mirror of
https://github.com/ollama/ollama.git
synced 2026-07-05 23:31:01 +00:00
1abd56b6e6
The batched MTP accept paths advance the cache by the whole accepted run before streaming it to the client. If the stream was cancelled partway (e.g. the caller disconnects), the loop returned before recording the remaining accepted tokens, leaving the cache offset ahead of session.outputs. close() then indexed the token log past its end and panicked with a slice-bounds error. Record the whole run to session.outputs before streaming any of it, so a cancelled stream can no longer desync the cache from the token log. The same bug is present on main, with identical mechanics: the accept paths there commit the cache to before+accepted and then stream in a loop that returns on cancellation before recording the rest. |
||
|---|---|---|
| .. | ||
| batch | ||
| cache | ||
| mlx | ||
| model | ||
| sample | ||
| cache.go | ||
| cache_test.go | ||
| cache_trie.go | ||
| cache_trie_test.go | ||
| client.go | ||
| imports.go | ||
| mtp.go | ||
| pipeline.go | ||
| runner.go | ||
| server.go | ||
| status_memory.go | ||
| status_memory_test.go | ||
| utf8_buffer.go | ||
| utf8_buffer_test.go | ||