ML//inference//local inference//Ollama//keep-alive

Keep-alive is the time a model server keeps a model loaded in memory after its last request before unloading it, and it decides whether the next request is answered at once or first waits for the whole model to be read back from disk. In Ollama the default is five minutes, set for the server with `OLLAMA_KEEP_ALIVE` or for one request with `keep_alive`.

Keep-alive is the time a model server keeps a model loaded in memory after its last request before unloading it, and it decides whether the next request is answered at once or first waits for the whole model to be read back from disk. In Ollama the default is five minutes, set for the server with OLLAMA_KEEP_ALIVE or for one request with keep_alive.

The trade is memory against waiting. While loaded and idle, the model holds its weights and its reserved KV cache (over 20 GB for a 35-billion-parameter model at 4.9 bits per weight) whether anyone uses it or not. Once it is unloaded, the next request pays the load: 55 seconds cold for that model on a laptop, and 33.8 seconds on a later load that found part of the file still in the operating system's cache.

For an interactive tool used in bursts, a keep-alive longer than the usual gap between uses (30 minutes, for instance) removes most cold loads; on a machine that needs its memory back for other work, a short one does the opposite.

A keep-alive of 0 unloads the model right after the request, which is how a benchmark makes every configuration start from a known state; a negative value keeps the model loaded indefinitely.

An unload also discards the cache that prefix caching would have reused, so the first request after it pays both the load and the full prefill.