mirror of
https://github.com/ollama/ollama.git
synced 2026-08-04 14:56:15 +00:00
4e96f4dbf2
Keeping the recurrent conv state small was handled unevenly: the committed live state was recopied on every commit — wasted work on single-token decode, where the window is already tiny — while boundary states captured as snapshots could still be plain slices of the forward-sized convolution buffer. A cached slice pins that whole buffer even though the trie's eviction accounting only counts the slice's bytes, so recurrent cache memory piled up across requests and eviction could never reclaim it. Compact each boundary state to its real size once, where it is produced in the conv wrapper, so live state and snapshots own only their own bytes and eviction sees the true cost. Single-token decode leaves the already-tiny window as a slice. Fixes #16698 |
||
|---|---|---|
| .. | ||
| cache.go | ||
| cache_test.go | ||
| kvcache.go | ||
| lazy_test.go | ||
| recurrent.go | ||
| recurrent_test.go | ||
| rotating.go | ||
| rotating_attention_test.go | ||
| rotating_multiturn_test.go | ||
| snapshot_capture_test.go | ||