Commit graph

3 commits

Author SHA1 Message Date
Daniel Hiltgen
cecac8bf1b
mlxrunner: add Inkling 2 Mini (#19799)
* mlxrunner: add Inkling 2 Mini

* mlx: fuse Inkling 2 Mini residual mixing on Metal

Fuse RMS normalization and query scoring in one kernel. Apply the
softmax weights in a second kernel to avoid storing per-query products.

* create: expose Inkling 2 Mini context length in model metadata

Copy model_max_length to max_position_embeddings during import so
ollama show reports the checkpoint's context length. Reject configs
that specify conflicting limits.

* mlxrunner: reduce Metal dispatches in Inkling 2 Mini decode

Reuse the prefill mask shader to compute relative position bias in one
dispatch for single-query attention. On M3 Ultra with Metal 3, compute
the loop controller's gain and forget projections in one matrix multiply.
2026-11-20 18:38:44 -08:00
Daniel Hiltgen
8be580ad09
mlx: add PPLX Decider v1.2 support (#19072)
Implement the Qwen3.5-backed decision scorer with noncausal attention,
the publisher readout, image handling, and System One prompt encoding.
2026-10-17 14:45:55 -07:00
Jesse Gross
2e036e7cdf mlx, mlxrunner: move the MLX engine out of x/
The MLX runner is the only Go inference runner left and is no longer
experimental, so its packages leave x/. The bindings become a top-level
mlx package beside the carried patches in mlx/compat, mirroring how
llama/ holds the llama.cpp integration, and the runner becomes mlxrunner
with the architectures nested under the package they implement.
Subpackages move with their parent unless listed.

  x/mlxrunner/mlx            mlx
  x/internal/mlxthread       mlx/mlxthread
  x/internal/mlxthreadtest   mlx/mlxthread/mlxthreadtest
  x/internal/mlxtest         mlx/mlxtest
  x/quant                    mlx/quant
  mlx/compat/*.patch         mlx/compat/mlx-c   (MLX patches go in mlx/compat/mlx)
  x/mlxrunner                mlxrunner
  x/models/nn                mlxrunner/nn
  x/models/<arch>            mlxrunner/model/<arch>
  x/mlxrunner/imports.go     mlxrunner/model/architectures   (new package)
  x/create                   create
  x/safetensors              fs/safetensors
  x/tokenizer                mlxrunner/tokenizer

Every package keeps its name, so the Go changes are the import path
rewrites the moves force, and the CMake, Dockerfile, CI cache keys, drift
check and Darwin payload script follow the new paths. Four edits are not
paths: the runner's blank architecture imports become the package
mlxrunner/model/architectures, so the list to extend for a new model sits
beside the architecture directories; a depguard rule keeps the two test
harnesses out of non-test code, as the x/internal placement used to; the
CI change filter's two entries for the long-deleted x/imagegen/mlx now
name the bindings' CMake project and the carried patches, so a change to
either builds the payload; and the tokenizer parity test reads its
fixtures from its own testdata instead of walking out of x/.

x/server and x/imagegen/manifest stay for the next two commits.
2026-09-16 14:06:08 -07:00
Renamed from x/create/classify.go (Browse further)