CERBERUS/ollama
1
0
Fork 0
mirror of https://github.com/ollama/ollama.git synced 2026-10-23 12:03:40 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions 3
ollama/server
History
Exact
Jesse Gross 5a0ff3116d server: apply structured outputs in a single pass on thinking models 
A format on a thinking model ran two generations: an unconstrained one,
cancelled once the parser reported content, then a re-rendered prompt with
the parsed thinking under the grammar. The restart cost a second prefill,
dropped the chunk that crossed the boundary, needed a harmony prompt hack,
stitched metrics across the two requests, and on MLX could leak a stray
first token into the JSON. The generate endpoint never deferred at all, so
its JSON was forced inside the thinking.

Both handlers now make one completion request that names the strings
ending the response's thinking, from the builtin parser or the generic
thinking parser, and the runner constrains only the content after them in
a single generation. The prompt is evaluated once and metrics pass straight
through. A raw generate prompt names no strings, since nothing says where
its response starts, and its format applies from the first token as
before. A format now applies to whatever follows the thinking, so a tool
call can no longer take the place of formatted content, which was already
the case with thinking off; harmony is the exception, since its tool calls
precede the final message.

The per-token metrics flag both runners carried for the cancelled first
pass has no caller left and goes with the two-pass code and its tests.

Fixes #18441
Fixes #17544
Fixes #14196
Fixes #10929
2026-09-22 11:24:11 -07:00
..
auth.go server: reject unexpected auth hosts (#13738) 2026-01-16 14:10:36 -05:00
auth_test.go server: reject unexpected auth hosts (#13738) 2026-01-16 14:10:36 -05:00
cloud_proxy.go openai: support web search in Responses API (#17686) 2026-08-12 11:51:54 -07:00
cloud_proxy_test.go cloud_proxy: for the web_search legacy path, flush on newlines (#14897) 2026-03-17 13:30:17 -07:00
codex_proxy.go app: add Ollama to ChatGPT Desktop (#18236) 2026-09-04 14:04:48 -07:00
codex_proxy_test.go app: harden Codex desktop proxy handling (#18244) 2026-09-04 17:48:55 -07:00
create.go mlx, mlxrunner: move the MLX engine out of x/ 2026-09-16 14:06:08 -07:00
create_fuzz_test.go create: add server-side MLX imports and drop GGUF conversion (#14969) 2026-09-14 20:32:36 -07:00
create_split_gguf.go create: add server-side MLX imports and drop GGUF conversion (#14969) 2026-09-14 20:32:36 -07:00
create_test.go create: add server-side MLX imports and drop GGUF conversion (#14969) 2026-09-14 20:32:36 -07:00
download.go lint fixes (#17897) 2026-08-20 10:23:25 -07:00
download_test.go server: detect download stalls before the first byte (#17259) 2026-07-21 11:28:19 -07:00
fixblobs.go
fixblobs_test.go
gemma4_test.go models: add support for gemma4-12b (#16457) 2026-06-03 07:44:57 -07:00
gguf_metadata.go create: add server-side MLX imports and drop GGUF conversion (#14969) 2026-09-14 20:32:36 -07:00
gguf_metadata_test.go tests: fix metadata deletion test flake (#18493) 2026-09-17 16:39:42 -07:00
glimmer_images_test.go Release v0.32.7 (#17646) 2026-08-10 04:04:56 -07:00
images.go server: allow registry cross-host redirects among allowlisted hosts (#18533) 2026-09-18 17:15:35 -07:00
images_test.go server: allow registry cross-host redirects among allowlisted hosts (#18533) 2026-09-18 17:15:35 -07:00
inference_request_log.go add ability to turn on debug request logging (#14106) 2026-03-19 17:08:17 -07:00
logprob.go logprob: add bytes to logprobs (#13068) 2025-11-13 13:49:25 -08:00
model.go create: add server-side MLX imports and drop GGUF conversion (#14969) 2026-09-14 20:32:36 -07:00
model_caches.go server: extract GGUF metadata and unify capabilities (#17858) 2026-09-10 11:16:37 -07:00
model_list.go create: add server-side MLX imports and drop GGUF conversion (#14969) 2026-09-14 20:32:36 -07:00
model_list_test.go nemotron_h: add MLX vision support (#17714) 2026-09-17 16:29:14 -07:00
model_recommendations.go app: add Ollama to ChatGPT Desktop (#18236) 2026-09-04 14:04:48 -07:00
model_recommendations_test.go app: add Ollama to ChatGPT Desktop (#18236) 2026-09-04 14:04:48 -07:00
model_resolver.go Reapply "don't require pulling stubs for cloud models" again (#14608) 2026-03-06 14:27:47 -08:00
model_resolver_test.go Reapply "don't require pulling stubs for cloud models" again (#14608) 2026-03-06 14:27:47 -08:00
model_show_cache.go api: expose model thinking levels and defaults (#18473) 2026-09-18 14:10:53 -07:00
model_show_cache_test.go server: remove OLLAMA_EXPERIMENT=client2 (#16962) 2026-07-06 13:15:39 -07:00
model_thinking.go api: expose model thinking levels and defaults (#18473) 2026-09-18 14:10:53 -07:00
model_thinking_test.go api: expose model thinking levels and defaults (#18473) 2026-09-18 14:10:53 -07:00
prompt.go server: align generate with native chat templates (#16878) 2026-06-24 13:43:56 -07:00
prompt_test.go anthropic: Preserve Claude local image-path tool results in renderer-owned prompt formatting (#16047) 2026-05-12 00:02:17 -04:00
renderer_resolution.go gemma4: image and audio input support 2026-09-02 15:07:28 -07:00
responses_compact.go server/openai: retry compaction after context overflow (#18324) 2026-09-08 14:42:03 -07:00
responses_compact_test.go openai: support standalone named function outputs (#18348) 2026-09-09 14:51:50 -07:00
routes.go server: apply structured outputs in a single pass on thinking models 2026-09-22 11:24:11 -07:00
routes_cloud_test.go Report cached prompt tokens (#17943) 2026-09-02 09:30:44 -07:00
routes_create_test.go mlx, mlxrunner: move the MLX engine out of x/ 2026-09-16 14:06:08 -07:00
routes_debug_test.go create: add server-side MLX imports and drop GGUF conversion (#14969) 2026-09-14 20:32:36 -07:00
routes_delete_test.go create: add server-side MLX imports and drop GGUF conversion (#14969) 2026-09-14 20:32:36 -07:00
routes_generate_renderer_test.go create: add server-side MLX imports and drop GGUF conversion (#14969) 2026-09-14 20:32:36 -07:00
routes_generate_test.go server: apply structured outputs in a single pass on thinking models 2026-09-22 11:24:11 -07:00
routes_harmony_streaming_test.go create: add server-side MLX imports and drop GGUF conversion (#14969) 2026-09-14 20:32:36 -07:00
routes_list_test.go server: extract GGUF metadata and unify capabilities (#17858) 2026-09-10 11:16:37 -07:00
routes_options_test.go Honor model generation defaults (#16471) 2026-09-01 14:52:07 -07:00
routes_parse_error_test.go create: add server-side MLX imports and drop GGUF conversion (#14969) 2026-09-14 20:32:36 -07:00
routes_request_log_test.go add ability to turn on debug request logging (#14106) 2026-03-19 17:08:17 -07:00
routes_test.go create: add server-side MLX imports and drop GGUF conversion (#14969) 2026-09-14 20:32:36 -07:00
routes_web_experimental_test.go openai: support web search in Responses API (#17686) 2026-08-12 11:51:54 -07:00
sched.go mlx, mlxrunner: move the MLX engine out of x/ 2026-09-16 14:06:08 -07:00
sched_test.go create: add server-side MLX imports and drop GGUF conversion (#14969) 2026-09-14 20:32:36 -07:00
show_safetensors.go mlxrunner, cmd, server: fold three small packages into their users 2026-09-16 14:06:08 -07:00
show_safetensors_test.go mlxrunner, cmd, server: fold three small packages into their users 2026-09-16 14:06:08 -07:00
sparse_common.go
sparse_windows.go
test_home_test.go app: harden update flows (#16100) 2026-05-11 12:24:01 -07:00
upload.go Clean up the manifest and modelpath (#13807) 2026-01-21 11:46:17 -08:00