CERBERUS/ollama
1
0
Fork 0
mirror of https://github.com/ollama/ollama.git synced 2026-10-22 19:41:46 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions 5
ollama/x/mlxrunner
History
Exact
Bruce MacDonald 6e65d95ef5
server: add cached eval metric to response 
Added cached prompt token counts to Ollama responses and compatibility usage fields.

This carries local `llama-server` `cache_n` and MLX cache hits through `/api/generate`, `/api/chat`, OpenAI-compatible endpoints, and Anthropic-compatible `/v1/messages`. Cloud responses are passed through as-is, so cache counts will show up there once Cloud starts returning them.
2026-06-25 16:21:08 -07:00
..
batch mlxrunner: support draft heads that maintain draft caches 2026-06-22 15:25:45 -07:00
cache mlxrunner: drive MTP speculation through cache snapshots 2026-06-09 00:39:19 -07:00
mlx llama-server followups (#16353) 2026-06-01 10:44:21 -07:00
model mlxrunner: choose the speculative draft length to maximize throughput 2026-06-22 15:25:45 -07:00
sample mlxrunner: apply in-flight drafts to proposal penalty history 2026-06-22 15:25:45 -07:00
cache.go mlxrunner: capture prefill snapshots across the forward 2026-06-09 00:39:19 -07:00
cache_test.go mlxrunner: capture prefill snapshots across the forward 2026-06-09 00:39:19 -07:00
cache_trie.go mlxrunner: drive MTP speculation through cache snapshots 2026-06-09 00:39:19 -07:00
cache_trie_test.go mlxrunner: share KV cache across conversations with common prefixes 2026-03-18 16:06:33 -07:00
client.go server: add cached eval metric to response 2026-06-25 16:21:08 -07:00
imports.go models: add cohere2_moe (Command A / North) to the MLX engine (#16670) 2026-06-16 23:15:21 -07:00
mtp.go mlxrunner: choose the speculative draft length to maximize throughput 2026-06-22 15:25:45 -07:00
mtp_test.go mlxrunner: choose the speculative draft length to maximize throughput 2026-06-22 15:25:45 -07:00
pipeline.go server: add cached eval metric to response 2026-06-25 16:21:08 -07:00
runner.go mlxrunner: support draft heads that maintain draft caches 2026-06-22 15:25:45 -07:00
server.go mlx: rework the MLX sampler (#16122) 2026-05-13 17:18:27 -07:00
speculate.go mlxrunner: choose the speculative draft length to maximize throughput 2026-06-22 15:25:45 -07:00
speculate_depth.go mlxrunner: choose the speculative draft length to maximize throughput 2026-06-22 15:25:45 -07:00
speculate_depth_test.go mlxrunner: choose the speculative draft length to maximize throughput 2026-06-22 15:25:45 -07:00
speculate_stats.go mlxrunner: choose the speculative draft length to maximize throughput 2026-06-22 15:25:45 -07:00
status_memory.go mlx: avoid status timeout during inference (#16086) 2026-05-11 16:03:38 -07:00
status_memory_test.go mlx: avoid status timeout during inference (#16086) 2026-05-11 16:03:38 -07:00
utf8_buffer.go consolidate the tokenizer (#14327) 2026-02-19 15:55:45 -08:00
utf8_buffer_test.go consolidate the tokenizer (#14327) 2026-02-19 15:55:45 -08:00