Get up and running with Kimi-K2.5, GLM-5, MiniMax, DeepSeek, gpt-oss, Qwen, Gemma and other models. https://ollama.com
Find a file
Daniel Hiltgen 64ee2f9847
model: add Laguna MLX support (#17237)
* model: add Laguna MLX support

Add Laguna XS 2, XS 2.1, and S 2.1 support to the MLX model and create paths.

Read the source config to apply one quantization policy across dense and routed MoE layers. Keep the tied output head and router at source precision, quantize supported attention and expert projections, selectively promote sensitive expert down projections, and emit per-tensor metadata for mixed quantization blobs.

Correct dense expert loading, BF16 source-layout handling, expert global-scale shapes and dtypes, routing-score scaling, and mixed-precision expert dispatch. Gate/up and down projections select quantized or dense execution independently so promoted BF16 down projections do not force quantized gate/up weights through the dense fallback.

Optimize the forward pass with compatible gate/up fusion, sorted standard GatherMM and GatherQMM operations for larger prefills, model-local mlx.Compile closures for elementwise MoE work, and cache-backed 512-token prefill chunks. This keeps the implementation on maintained MLX operations without custom kernels.

Add focused tests for Laguna configuration variants, quantization policy and metadata, dense and routed expert loading, mixed-precision dispatch, compiled-versus-eager parity, fused projections, routing, and prefill chunking.

* review comments and S 2.1 performance fixes

Address renderer/parser selection and mixed-precision expert quantization review feedback.

Keep Laguna weights resident on Metal to prevent repeated paging of its large, sparsely accessed expert buffers. Scope this policy to Laguna GPU execution.

Remove obsolete 512-token prefill chunking now that the runner's 2048-token path is faster.

* review comments addressed

* fix create
2026-07-24 18:24:53 -07:00
.github CI: fix missing CUDA v13.4 sub-package (#17288) 2026-07-21 12:25:10 -07:00
agent cmd/tui: agent system prompt command (#17296) 2026-07-24 14:44:25 -07:00
anthropic anthropic: close text block before starting thinking block (#17225) 2026-07-17 10:24:38 -04:00
api docs: add renderer/parser fields to the API docs (#17275) 2026-07-20 16:22:46 -07:00
app test: harden flaky updater and transfer unit tests (#17378) 2026-07-24 13:23:30 -07:00
auth auth: fix problems with the ollama keypairs (#12373) 2025-09-22 23:20:20 -07:00
cmake mlx: restore NAX in Metal v4 builds (#17160) 2026-07-13 12:50:38 -07:00
cmd cmd/tui: agent system prompt command (#17296) 2026-07-24 14:44:25 -07:00
convert llama: default qwen2.5vl window attention metadata (#16868) 2026-06-24 10:35:29 -07:00
discover discover: fall back to standard CUDA when the JetPack runner is absent (#16949) 2026-07-02 08:34:35 -07:00
docs docs: update retirements (#17289) 2026-07-22 14:24:11 -07:00
envconfig runner: Remove CGO engines, use llama-server exclusively for GGML models (#16031) 2026-05-29 13:35:47 -07:00
format chore(all): replace instances of interface with any (#10067) 2025-04-02 09:44:27 -07:00
fs create: harden GGUF create flows (#17062) 2026-07-06 16:20:20 -07:00
harmony runner: Remove CGO engines, use llama-server exclusively for GGML models (#16031) 2026-05-29 13:35:47 -07:00
integration model: align Laguna with upstream llama.cpp (#17335) 2026-07-22 17:09:18 -07:00
internal Reapply "don't require pulling stubs for cloud models" again (#14608) 2026-03-06 14:27:47 -08:00
llama model: align Laguna with upstream llama.cpp (#17335) 2026-07-22 17:09:18 -07:00
llm llama: enable dio on linux CUDA/ROCm iGPUs (#17286) 2026-07-21 10:53:08 -07:00
logutil runner: Remove CGO engines, use llama-server exclusively for GGML models (#16031) 2026-05-29 13:35:47 -07:00
manifest runner: Remove CGO engines, use llama-server exclusively for GGML models (#16031) 2026-05-29 13:35:47 -07:00
middleware Add support for gemma4 (#15214) 2026-04-02 11:33:33 -07:00
ml llama: enable FA on CUDA CC 6.x GPUs (#16994) 2026-07-02 17:11:39 -07:00
model model: align Laguna with upstream llama.cpp (#17335) 2026-07-22 17:09:18 -07:00
openai openai: align models list with tags (#16556) 2026-06-05 17:59:05 -07:00
parser model: add Laguna v8 chat support and fix Metal inference (#17291) 2026-07-21 16:06:29 -07:00
progress Add z-image image generation prototype (#13659) 2026-01-09 21:09:46 -08:00
readline runner: Remove CGO engines, use llama-server exclusively for GGML models (#16031) 2026-05-29 13:35:47 -07:00
runner runner: Remove CGO engines, use llama-server exclusively for GGML models (#16031) 2026-05-29 13:35:47 -07:00
scripts win: support CUDA on Windows ARM64 (#16931) 2026-07-21 10:53:30 -07:00
server server: fix ps data race on scheduler loaded map (#17376) 2026-07-24 13:23:49 -07:00
template runner: Remove CGO engines, use llama-server exclusively for GGML models (#16031) 2026-05-29 13:35:47 -07:00
thinking thinking: fix double emit when no opening tag 2025-08-21 21:03:12 -07:00
tokenizer tokenizer: fix multi-regex BPE offset handling (#15844) 2026-04-27 14:14:27 -07:00
tools tools: ignore braces inside JSON strings when detecting tool call end (#16937) 2026-06-27 12:00:55 -07:00
types mlx: Gemma4 MTP speculative decoding (#15980) 2026-05-05 08:55:04 -07:00
version add version 2023-08-22 09:40:58 -07:00
x model: add Laguna MLX support (#17237) 2026-07-24 18:24:53 -07:00
.dockerignore next build (#8539) 2025-01-29 15:03:38 -08:00
.gitattributes .gitattributes: add app/webview to linguist-vendored (#13274) 2025-11-29 23:46:10 -05:00
.gitignore create: Clean up experimental paths, fix create from existing safetensor model (#14679) 2026-04-07 08:12:57 -07:00
.golangci.yaml ci: restore previous linter rules (#13322) 2025-12-03 18:55:02 -08:00
AGENTS.md Add AGENTS.md and CLAUDE.md to root repository (#16604) 2026-06-07 10:57:59 -07:00
CLAUDE.md Add AGENTS.md and CLAUDE.md to root repository (#16604) 2026-06-07 10:57:59 -07:00
CMakeLists.txt runner: Remove CGO engines, use llama-server exclusively for GGML models (#16031) 2026-05-29 13:35:47 -07:00
CMakePresets.json runner: Remove CGO engines, use llama-server exclusively for GGML models (#16031) 2026-05-29 13:35:47 -07:00
CONTRIBUTING.md docs: fix typos in repository documentation (#10683) 2025-11-15 20:22:29 -08:00
Dockerfile build: bump Linux toolchain to GCC 13 (#17244) 2026-07-20 11:54:39 -07:00
go.mod server: remove OLLAMA_EXPERIMENT=client2 (#16962) 2026-07-06 13:15:39 -07:00
go.sum server: remove OLLAMA_EXPERIMENT=client2 (#16962) 2026-07-06 13:15:39 -07:00
LICENSE proto -> ollama 2023-06-26 15:57:13 -04:00
LLAMA_CPP_VERSION model: align Laguna with upstream llama.cpp (#17335) 2026-07-22 17:09:18 -07:00
main.go lint 2024-08-01 17:06:06 -07:00
MLX_C_VERSION Update MLX and MLX-C with threading fixes (#15845) 2026-05-03 10:03:14 -07:00
MLX_VERSION mlx update (#17332) 2026-07-22 13:36:49 -07:00
README.md docs: update docs examples to use Gemma 4 instead of Gemma 3 (#16607) 2026-06-07 12:43:13 -07:00
SECURITY.md docs: fix typos in repository documentation (#10683) 2025-11-15 20:22:29 -08:00

ollama

Ollama

Start building with open models.

Download

macOS

curl -fsSL https://ollama.com/install.sh | sh

or download manually

Windows

irm https://ollama.com/install.ps1 | iex

or download manually

Linux

curl -fsSL https://ollama.com/install.sh | sh

Manual install instructions

Docker

The official Ollama Docker image ollama/ollama is available on Docker Hub.

Libraries

Community

Get started

ollama

You'll be prompted to run a model or connect Ollama to your existing agents or applications such as Claude Code, OpenClaw, OpenCode , Codex, Copilot, and more.

Coding

To launch a specific integration:

ollama launch claude

Supported integrations include Claude Code, Codex, Copilot CLI, Droid, and OpenCode.

AI assistant

Use OpenClaw to turn Ollama into a personal AI assistant across WhatsApp, Telegram, Slack, Discord, and more:

ollama launch openclaw

Chat with a model

Run and chat with Gemma 4:

ollama run gemma4

See ollama.com/library for the full list.

See the quickstart guide for more details.

REST API

Ollama has a REST API for running and managing models.

curl http://localhost:11434/api/chat -d '{
  "model": "gemma4",
  "messages": [{
    "role": "user",
    "content": "Why is the sky blue?"
  }],
  "stream": false
}'

See the API documentation for all endpoints.

Python

pip install ollama
from ollama import chat

response = chat(model='gemma4', messages=[
  {
    'role': 'user',
    'content': 'Why is the sky blue?',
  },
])
print(response.message.content)

JavaScript

npm i ollama
import ollama from "ollama";

const response = await ollama.chat({
  model: "gemma4",
  messages: [{ role: "user", content: "Why is the sky blue?" }],
});
console.log(response.message.content);

Supported backends

  • llama.cpp project founded by Georgi Gerganov.

Documentation

Community Integrations

Want to add your project? Open a pull request.

Chat Interfaces

Web

Desktop

  • Dify.AI - LLM app development platform
  • AnythingLLM - All-in-one AI app for Mac, Windows, and Linux
  • Maid - Cross-platform mobile and desktop client
  • Witsy - AI desktop app for Mac, Windows, and Linux
  • Cherry Studio - Multi-provider desktop client
  • Ollama App - Multi-platform client for desktop and mobile
  • PyGPT - AI desktop assistant for Linux, Windows, and Mac
  • Alpaca - GTK4 client for Linux and macOS
  • SwiftChat - Cross-platform including iOS, Android, and Apple Vision Pro
  • Enchanted - Native macOS and iOS client
  • RWKV-Runner - Multi-model desktop runner
  • Ollama Grid Search - Evaluate and compare models
  • macai - macOS client for Ollama and ChatGPT
  • AI Studio - Multi-provider desktop IDE
  • Reins - Parameter tuning and reasoning model support
  • ConfiChat - Privacy-focused with optional encryption
  • LLocal.in - Electron desktop client
  • MindMac - AI chat client for Mac
  • Msty - Multi-model desktop client
  • BoltAI for Mac - AI chat client for Mac
  • IntelliBar - AI-powered assistant for macOS
  • Kerlig AI - AI writing assistant for macOS
  • Hillnote - Markdown-first AI workspace
  • Perfect Memory AI - Productivity AI personalized by screen and meeting history

Mobile

SwiftChat, Enchanted, Maid, Ollama App, Reins, and ConfiChat listed above also support mobile platforms.

Code Editors & Development

Libraries & SDKs

Frameworks & Agents

RAG & Knowledge Bases

  • RAGFlow - RAG engine based on deep document understanding
  • R2R - Open-source RAG engine
  • MaxKB - Ready-to-use RAG chatbot
  • Minima - On-premises or fully local RAG
  • Chipper - AI interface with Haystack RAG
  • ARGO - RAG and deep research on Mac/Windows/Linux
  • Archyve - RAG-enabling document library
  • Casibase - AI knowledge base with RAG and SSO
  • BrainSoup - Native client with RAG and multi-agent automation

Bots & Messaging

Terminal & CLI

Productivity & Apps

Observability & Monitoring

  • Opik - Debug, evaluate, and monitor LLM applications
  • OpenLIT - OpenTelemetry-native monitoring for Ollama and GPUs
  • Lunary - LLM observability with analytics and PII masking
  • Langfuse - Open source LLM observability
  • HoneyHive - AI observability and evaluation for agents
  • MLflow Tracing - Open source LLM observability

Database & Embeddings

Infrastructure & Deployment

Cloud

Package Managers