ollama/integration
Daniel Hiltgen cecac8bf1b
mlxrunner: add Inkling 2 Mini (#19799)
* mlxrunner: add Inkling 2 Mini

* mlx: fuse Inkling 2 Mini residual mixing on Metal

Fuse RMS normalization and query scoring in one kernel. Apply the
softmax weights in a second kernel to avoid storing per-query products.

* create: expose Inkling 2 Mini context length in model metadata

Copy model_max_length to max_position_embeddings during import so
ollama show reports the checkpoint's context length. Reject configs
that specify conflicting limits.

* mlxrunner: reduce Metal dispatches in Inkling 2 Mini decode

Reuse the prefill mask shader to compute relative position bias in one
dispatch for single-query attention. On M3 Ultra with Metal 3, compute
the loop controller's gain and forget projections in one matrix multiply.
2026-11-20 18:38:44 -08:00
..
testdata tests: fix embeddinggemma integration test (#12830) 2025-10-29 11:07:28 -07:00
api_test.go fix(docs): correct typos found during code review (#17579) 2026-08-31 08:59:55 -07:00
audio_test.go mlx: add LFM3 and d1 models (#19333) 2026-10-27 08:47:18 -07:00
audio_test_data_test.go Add support for gemma4 (#15214) 2026-04-02 11:33:33 -07:00
basic_test.go test: revamp integration test entrpoints (#16560) 2026-07-21 16:06:38 -07:00
chat_cases_test.go test: revamp integration test entrpoints (#16560) 2026-07-21 16:06:38 -07:00
compatibility_model_matrix_test.go llama-server: prepare to remove compatibility patch 2026-09-24 19:25:52 -07:00
compatibility_models_test.go llama-server: prepare to remove compatibility patch 2026-09-24 19:25:52 -07:00
concurrency_test.go mlxrunner: choose default parallelism from available memory (#19785) 2026-11-20 07:13:42 -08:00
concurrent_endpoints_test.go mlx: support concurrent generation (#19197) 2026-11-10 13:24:12 -08:00
context_test.go mlx: support concurrent generation (#19197) 2026-11-10 13:24:12 -08:00
create_test.go create: add server-side MLX imports and drop GGUF conversion (#14969) 2026-09-14 20:32:36 -07:00
embed_cases_test.go test: revamp integration test entrpoints (#16560) 2026-07-21 16:06:38 -07:00
embed_test.go mlx: support concurrent generation (#19197) 2026-11-10 13:24:12 -08:00
generate_jinja_test.go server: align generate with native chat templates (#16878) 2026-06-24 13:43:56 -07:00
llm_image_test.go llm: transcode WebP images for llama-server (#17755) 2026-08-14 13:21:11 -07:00
max_queue_test.go test: revamp integration test entrpoints (#16560) 2026-07-21 16:06:38 -07:00
migration_test.go server: skip local model compatibility migration (#18908) 2026-10-09 10:36:11 -07:00
mlx_batching_test.go mlxrunner: add Inkling 2 Mini (#19799) 2026-11-20 18:38:44 -08:00
quantization_test.go test: split create integration tests out (#18470) 2026-10-05 15:01:48 -07:00
README.md test: revamp integration test entrpoints (#16560) 2026-07-21 16:06:38 -07:00
reg_create_test.go test: split create integration tests out (#18470) 2026-10-05 15:01:48 -07:00
reg_fast_test.go mlx: support concurrent generation (#19197) 2026-11-10 13:24:12 -08:00
reg_groups_test.go mlx: support concurrent generation (#19197) 2026-11-10 13:24:12 -08:00
reg_library_test.go test: revamp integration test entrpoints (#16560) 2026-07-21 16:06:38 -07:00
reg_release_test.go mlx: support concurrent generation (#19197) 2026-11-10 13:24:12 -08:00
reg_required_test.go test: split create integration tests out (#18470) 2026-10-05 15:01:48 -07:00
reg_test.go test: revamp integration test entrpoints (#16560) 2026-07-21 16:06:38 -07:00
structured_output_test.go mlxrunner: add structured output support 2026-08-25 17:14:23 -07:00
systemone_test.go mlx: support concurrent generation (#19197) 2026-11-10 13:24:12 -08:00
thinking_test.go model: align Laguna with upstream llama.cpp (#17335) 2026-07-22 17:09:18 -07:00
tools_routes_test.go renderers/qwen: tolerate non-leading system messages (#17757) 2026-08-14 14:12:50 -07:00
tools_stress_test.go test: revamp integration test entrpoints (#16560) 2026-07-21 16:06:38 -07:00
tools_test.go llama-server: prepare to remove compatibility patch 2026-09-24 19:25:52 -07:00
utils_test.go llama-server: prepare to remove compatibility patch 2026-09-24 19:25:52 -07:00
vision_ocr_test.go mlxrunner: add Inkling 2 Mini (#19799) 2026-11-20 18:38:44 -08:00
vision_test.go test: revamp integration test entrpoints (#16560) 2026-07-21 16:06:38 -07:00
vision_test_data_test.go Add support for gemma4 (#15214) 2026-04-02 11:33:33 -07:00

Integration Tests

This directory contains integration tests to exercise Ollama end-to-end to verify behavior

By default, these tests are disabled so go test ./... will exercise only unit tests. To run integration tests, pass the integration tag and one of the scoped tags:

go test -tags=integration,fast -v -count 1 ./integration/
go test -tags=integration,release -v -count 1 -timeout 30m ./integration/
go test -tags=integration,library -v -count 1 -timeout 120m ./integration/

Tags:

  • fast: quick runner/model smoke coverage.
  • release: release regression coverage.
  • library: broad library coverage requiring about 2.5 TiB of disk space.

Scope wiring and model selections live in integration/reg_fast_test.go, integration/reg_release_test.go, and integration/reg_library_test.go.

The integration tests have 2 modes of operating.

  1. By default, on Unix systems, they will start the server on a random port, run the tests, and then shutdown the server. On Windows you must ALWAYS run the server on OLLAMA_HOST for the tests to work.
  2. If OLLAMA_TEST_EXISTING is set to a non-empty string, the tests will run against an existing running server, which can be remote based on your OLLAMA_HOST environment variable

Set OLLAMA_TEST_LOG_SERVER=1 to print the managed server log after each test run, even when the tests pass. This only applies when the integration test harness starts the server.

Important

Before running the tests locally without the "test existing" setting, compile ollama from the top of the source tree go build . in addition to GPU support with cmake if applicable on your platform. The integration tests expect to find an ollama binary at the top of the tree.

Testing a New Model

When implementing new model architecture, use OLLAMA_TEST_MODEL to run the integration suite against your model with either the fast or release coverage.

# Build the binary first
go build .

# Run integration tests against it
OLLAMA_TEST_MODEL=mymodel go test -tags=integration,fast -v -count 1 ./integration/