ollama/x/mlxrunner/batch
Jesse Gross 2f84872ce0 mlxrunner: return the draft-conditioning state from a model forward
A draft model conditions on state that the target produces during its
own forward pass. For an MTP head or an assistant model that state is
the final hidden state; for a block draft it is the concatenated
outputs of several layers. The choice belongs to the model, so Forward
now returns the conditioning state along with the hidden state to
unembed. Models without a special conditioning state return the final
hidden state for both, and the decode paths hand the value to the
drafter without looking at it.
2026-08-07 19:33:35 -07:00
..
batch.go mlxrunner: return the draft-conditioning state from a model forward 2026-08-07 19:33:35 -07:00