mirror of
https://github.com/ollama/ollama.git
synced 2026-07-11 02:04:06 +00:00
15e6076d79
This change adds support for MTP (multi-token prediction) speculative decoding for the gemma4 model family. It includes: * support for importing safetensors based gemma4 draft models with `ollama create` * a new DRAFT command in the Modelfile for specifying draft models * a --quantize-draft flag for the ollama create command to quantize the draft model * cache support for speculation * changes to the rotating cache to be able to handle MTP correctly * sampling support for draft model token prediction --------- Co-authored-by: Daniel Hiltgen <daniel@ollama.com> |
||
|---|---|---|
| .. | ||
| base | ||
| embedding.go | ||
| embedding_test.go | ||
| linear.go | ||
| quant.go | ||
| root.go | ||