CERBERUS/ollama
1
0
Fork 0
mirror of https://github.com/ollama/ollama.git synced 2026-10-20 18:41:17 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions

llm: make load time stall duration configurable via OLLAMA_LOAD_TIMEOUT

Browse source 
With the new very large parameter models, some users are willing to wait for
a very long time for models to load.
This commit is contained in:
Daniel Hiltgen 2024-09-05 14:00:08 -07:00 • committed by GitHub
parent b05c9e83d9
commit 6719097649
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
4 changed files with 60 additions and 7 deletions
Download patch file Download diff file Expand all files Collapse all files

1
cmd/cmd.go
View file

@ -1422,6 +1422,7 @@ func NewCLI() *cobra.Command {
envVars["OLLAMA_FLASH_ATTENTION"],
envVars["OLLAMA_LLM_LIBRARY"],
envVars["OLLAMA_GPU_OVERHEAD"],
envVars["OLLAMA_LOAD_TIMEOUT"],
})
default:
appendEnvDocs(cmd, envs)