ollama/mlxrunner/cache
Daniel Hiltgen cecac8bf1b
mlxrunner: add Inkling 2 Mini (#19799)
* mlxrunner: add Inkling 2 Mini

* mlx: fuse Inkling 2 Mini residual mixing on Metal

Fuse RMS normalization and query scoring in one kernel. Apply the
softmax weights in a second kernel to avoid storing per-query products.

* create: expose Inkling 2 Mini context length in model metadata

Copy model_max_length to max_position_embeddings during import so
ollama show reports the checkpoint's context length. Reject configs
that specify conflicting limits.

* mlxrunner: reduce Metal dispatches in Inkling 2 Mini decode

Reuse the prefill mask shader to compute relative position bias in one
dispatch for single-query attention. On M3 Ultra with Metal 3, compute
the loop controller's gain and forget projections in one matrix multiply.
2026-11-20 18:38:44 -08:00
..
batch.go mlx: support concurrent generation (#19197) 2026-11-10 13:24:12 -08:00
batch_test.go mlx: support concurrent generation (#19197) 2026-11-10 13:24:12 -08:00
cache.go mlxrunner: adapt saved history to memory pressure (#19678) 2026-11-19 15:41:42 -08:00
cache_test.go mlx, mlxrunner: move the MLX engine out of x/ 2026-09-16 14:06:08 -07:00
hidden.go mlx: System one support (#18701) 2026-10-03 13:17:21 -07:00
hidden_test.go mlx: System one support (#18701) 2026-10-03 13:17:21 -07:00
kvcache.go mlxrunner: adapt saved history to memory pressure (#19678) 2026-11-19 15:41:42 -08:00
lazy_test.go mlxrunner: adapt saved history to memory pressure (#19678) 2026-11-19 15:41:42 -08:00
memory.go mlxrunner: choose default parallelism from available memory (#19785) 2026-11-20 07:13:42 -08:00
memory_test.go mlxrunner: choose default parallelism from available memory (#19785) 2026-11-20 07:13:42 -08:00
recurrent.go mlx: support concurrent generation (#19197) 2026-11-10 13:24:12 -08:00
recurrent_test.go mlx: compact restored recurrent state after speculative rollback (#18955) 2026-10-12 09:28:22 -07:00
retention_test.go mlxrunner: adapt saved history to memory pressure (#19678) 2026-11-19 15:41:42 -08:00
rotating.go mlxrunner: add Inkling 2 Mini (#19799) 2026-11-20 18:38:44 -08:00
rotating_attention_test.go MLX, MLX-C: version bump - prompt speed up for gemma/flint/laguna (#19291) 2026-10-25 21:25:58 -07:00
rotating_multiturn_test.go mlxrunner: add Inkling 2 Mini (#19799) 2026-11-20 18:38:44 -08:00
snapshot_capture_test.go mlx: support concurrent generation (#19197) 2026-11-10 13:24:12 -08:00