mirror of
https://github.com/ollama/ollama.git
synced 2026-07-11 10:14:28 +00:00
d00622060f
Speculation used a parallel hierarchy of wrapper cache types that shadowed the live caches and reconciled against them on commit. Replace it with snapshot/restore on the live caches themselves: a cache snapshots itself as a write crosses each offset, and the runner commits a batched draft by restoring to the accepted count. The wrappers and the comparison plumbing around them are gone. Snapshots are lazy. A KV or rotating capture indexes into the live buffer and owns no memory until a destructive write forces a copy-out, so rejecting a draft is free. Recurrent layers now validate in the same batched pass rather than falling back to serial. A gated-delta layer reports its interior split offsets and hands back the recurrent state at each one, which the cache records as a snapshot. |
||
|---|---|---|
| .. | ||
| assistant.go | ||
| gemma4.go | ||
| gemma4_moe_test.go | ||
| gemma4_test.go | ||