mirror of
https://github.com/ollama/ollama.git
synced 2026-10-20 18:41:17 +00:00
Get up and running with Kimi-K2.5, GLM-5, MiniMax, DeepSeek, gpt-oss, Qwen, Gemma and other models.
https://ollama.com
1ce8a6b26d
Two QwenVL families that ship with monolithic vision+text Ollama GGUFs.
qwen2.5-vl (text + clip):
* Text: arch translation qwen25vl→qwen2vl, KV prefix copy, mrope_section
(3 elems) → rope.dimension_sections (4 elems, padded with 0).
* Clip: tensor renames (v.merger.* → v.post_ln/mm.0/mm.2,
v.patch_embd_{0,1} → v.patch_embd.weight{,.1}), use_silu, derive
n_wa_pattern from fullatt_block_indexes[0]+1, image_size=560 default,
standard CLIP image_mean/std, F32 promote of patch_embd for Metal.
qwen3-vl (text + clip):
* Text: inject qwen3vl.rope.dimension_sections=[24,20,20,0] (Qwen3-VL-8B
HF default — head_dim=128, sum=64) and qwen3vl.n_deepstack_layers
derived from deepstack_visual_indexes array length.
* Clip: per-block QKV merge (Ollama stores separate Q/K/V; upstream
qwen3vl_merger graph reads combined attn_qkv); deepstack remap
v.deepstack_merger.X.* → v.deepstack.{indexes[X]}.* (with
linear_fc{1,2} → fc{1,2}); merger renames matching upstream LLaVA
proj layout (v.post_ln + mm.0/mm.2); patch_embed split (16x16x2
Conv3D → two 16x16 Conv2Ds, F16→F32); per-block substring renames
(norm1/2 → ln1/2, mlp.linear_fc1/2 → ffn_up/down); F32 promote of
position_embd.
The qwen3-vl QKV merge needed a new util helper
(register_concat_load_to_f32) because the Ollama blob mixes types within
a single block (F16 Q/K + Q8_0 V) — the existing byte-concat
register_concat_load only works when all sources share a type. The new
helper dequantizes each source to F32 via ggml_get_type_traits->to_float
and concatenates; caller sets the destination tensor type to F32.
Verified end-to-end via `ollama run`:
* qwen2.5vl: image of NYT moon-landing front page → correct caption.
* qwen3-vl: same image → "New York Times front page from July 21,
1969, headlines the Apollo 11 moon landing with 'MEN WALK ON MOON'
and details of astronauts collecting rocks and planting a flag."
Also added qwen25vl + qwen3vl to the compatClipArches allowlist in
llm/llama_server.go so the auto-mmproj path activates for these
monolithic blobs (same mechanism as gemma3/4/qwen35moe/etc.).
|
||
|---|---|---|
| .github | ||
| anthropic | ||
| api | ||
| app | ||
| auth | ||
| cmd | ||
| convert | ||
| discover | ||
| docs | ||
| envconfig | ||
| format | ||
| fs | ||
| harmony | ||
| integration | ||
| internal | ||
| kvcache | ||
| llama | ||
| llm | ||
| logutil | ||
| manifest | ||
| middleware | ||
| ml | ||
| model | ||
| openai | ||
| parser | ||
| progress | ||
| readline | ||
| runner | ||
| scripts | ||
| server | ||
| template | ||
| thinking | ||
| tokenizer | ||
| tools | ||
| types | ||
| version | ||
| x | ||
| .dockerignore | ||
| .gitattributes | ||
| .gitignore | ||
| .golangci.yaml | ||
| CMakeLists.txt | ||
| CMakePresets.json | ||
| CONTRIBUTING.md | ||
| Dockerfile | ||
| go.mod | ||
| go.sum | ||
| LICENSE | ||
| LLAMA_CPP_VERSION | ||
| main.go | ||
| MLX_C_VERSION | ||
| MLX_VERSION | ||
| README.md | ||
| SECURITY.md | ||
Ollama
Start building with open models.
Download
macOS
curl -fsSL https://ollama.com/install.sh | sh
Windows
irm https://ollama.com/install.ps1 | iex
Linux
curl -fsSL https://ollama.com/install.sh | sh
Docker
The official Ollama Docker image ollama/ollama is available on Docker Hub.
Libraries
Community
Get started
ollama
You'll be prompted to run a model or connect Ollama to your existing agents or applications such as Claude Code, OpenClaw, OpenCode , Codex, Copilot, and more.
Coding
To launch a specific integration:
ollama launch claude
Supported integrations include Claude Code, Codex, Copilot CLI, Droid, and OpenCode.
AI assistant
Use OpenClaw to turn Ollama into a personal AI assistant across WhatsApp, Telegram, Slack, Discord, and more:
ollama launch openclaw
Chat with a model
Run and chat with Gemma 3:
ollama run gemma3
See ollama.com/library for the full list.
See the quickstart guide for more details.
REST API
Ollama has a REST API for running and managing models.
curl http://localhost:11434/api/chat -d '{
"model": "gemma3",
"messages": [{
"role": "user",
"content": "Why is the sky blue?"
}],
"stream": false
}'
See the API documentation for all endpoints.
Python
pip install ollama
from ollama import chat
response = chat(model='gemma3', messages=[
{
'role': 'user',
'content': 'Why is the sky blue?',
},
])
print(response.message.content)
JavaScript
npm i ollama
import ollama from "ollama";
const response = await ollama.chat({
model: "gemma3",
messages: [{ role: "user", content: "Why is the sky blue?" }],
});
console.log(response.message.content);
Supported backends
- llama.cpp project founded by Georgi Gerganov.
Documentation
Community Integrations
Want to add your project? Open a pull request.
Chat Interfaces
Web
- Open WebUI - Extensible, self-hosted AI interface
- Onyx - Connected AI workspace
- LibreChat - Enhanced ChatGPT clone with multi-provider support
- Lobe Chat - Modern chat framework with plugin ecosystem (docs)
- NextChat - Cross-platform ChatGPT UI (docs)
- Perplexica - AI-powered search engine, open-source Perplexity alternative
- big-AGI - AI suite for professionals
- Lollms WebUI - Multi-model web interface
- ChatOllama - Chatbot with knowledge bases
- Bionic GPT - On-premise AI platform
- Chatbot UI - ChatGPT-style web interface
- Hollama - Minimal web interface
- Chatbox - Desktop and web AI client
- chat - Chat web app for teams
- Ollama RAG Chatbot - Chat with multiple PDFs using RAG
- Tkinter-based client - Python desktop client
Desktop
- Dify.AI - LLM app development platform
- AnythingLLM - All-in-one AI app for Mac, Windows, and Linux
- Maid - Cross-platform mobile and desktop client
- Witsy - AI desktop app for Mac, Windows, and Linux
- Cherry Studio - Multi-provider desktop client
- Ollama App - Multi-platform client for desktop and mobile
- PyGPT - AI desktop assistant for Linux, Windows, and Mac
- Alpaca - GTK4 client for Linux and macOS
- SwiftChat - Cross-platform including iOS, Android, and Apple Vision Pro
- Enchanted - Native macOS and iOS client
- RWKV-Runner - Multi-model desktop runner
- Ollama Grid Search - Evaluate and compare models
- macai - macOS client for Ollama and ChatGPT
- AI Studio - Multi-provider desktop IDE
- Reins - Parameter tuning and reasoning model support
- ConfiChat - Privacy-focused with optional encryption
- LLocal.in - Electron desktop client
- MindMac - AI chat client for Mac
- Msty - Multi-model desktop client
- BoltAI for Mac - AI chat client for Mac
- IntelliBar - AI-powered assistant for macOS
- Kerlig AI - AI writing assistant for macOS
- Hillnote - Markdown-first AI workspace
- Perfect Memory AI - Productivity AI personalized by screen and meeting history
Mobile
- Ollama Android Chat - One-click Ollama on Android
SwiftChat, Enchanted, Maid, Ollama App, Reins, and ConfiChat listed above also support mobile platforms.
Code Editors & Development
- Cline - VS Code extension for multi-file/whole-repo coding
- Continue - Open-source AI code assistant for any IDE
- Void - Open source AI code editor, Cursor alternative
- Copilot for Obsidian - AI assistant for Obsidian
- twinny - Copilot and Copilot chat alternative
- gptel Emacs client - LLM client for Emacs
- Ollama Copilot - Use Ollama as GitHub Copilot
- Obsidian Local GPT - Local AI for Obsidian
- Ellama Emacs client - LLM tool for Emacs
- orbiton - Config-free text editor with Ollama tab completion
- AI ST Completion - Sublime Text 4 AI assistant
- VT Code - Rust-based terminal coding agent with Tree-sitter
- QodeAssist - AI coding assistant for Qt Creator
- AI Toolkit for VS Code - Microsoft-official VS Code extension
- Open Interpreter - Natural language interface for computers
Libraries & SDKs
- LiteLLM - Unified API for 100+ LLM providers
- Semantic Kernel - Microsoft AI orchestration SDK
- LangChain4j - Java LangChain (example)
- LangChainGo - Go LangChain (example)
- Spring AI - Spring framework AI support (docs)
- LangChain and LangChain.js with example
- Ollama for Ruby - Ruby LLM library
- any-llm - Unified LLM interface by Mozilla
- OllamaSharp for .NET - .NET SDK
- LangChainRust - Rust LangChain (example)
- Agents-Flex for Java - Java agent framework (example)
- Elixir LangChain - Elixir LangChain
- Ollama-rs for Rust - Rust SDK
- LangChain for .NET - .NET LangChain (example)
- chromem-go - Go vector database with Ollama embeddings (example)
- LangChainDart - Dart LangChain
- LlmTornado - Unified C# interface for multiple inference APIs
- Ollama4j for Java - Java SDK
- Ollama for Laravel - Laravel integration
- Ollama for Swift - Swift SDK
- LlamaIndex and LlamaIndexTS - Data framework for LLM apps
- Haystack - AI pipeline framework
- Firebase Genkit - Google AI framework
- Ollama-hpp for C++ - C++ SDK
- PromptingTools.jl - Julia LLM toolkit (example)
- Ollama for R - rollama - R SDK
- Portkey - AI gateway
- Testcontainers - Container-based testing
- LLPhant - PHP AI framework
Frameworks & Agents
- AutoGPT - Autonomous AI agent platform
- crewAI - Multi-agent orchestration framework
- Strands Agents - Model-driven agent building by AWS
- Cheshire Cat - AI assistant framework
- any-agent - Unified agent framework interface by Mozilla
- Stakpak - Open source DevOps agent
- Hexabot - Conversational AI builder
- Neuro SAN - Multi-agent orchestration (docs)
RAG & Knowledge Bases
- RAGFlow - RAG engine based on deep document understanding
- R2R - Open-source RAG engine
- MaxKB - Ready-to-use RAG chatbot
- Minima - On-premises or fully local RAG
- Chipper - AI interface with Haystack RAG
- ARGO - RAG and deep research on Mac/Windows/Linux
- Archyve - RAG-enabling document library
- Casibase - AI knowledge base with RAG and SSO
- BrainSoup - Native client with RAG and multi-agent automation
Bots & Messaging
- LangBot - Multi-platform messaging bots with agents and RAG
- AstrBot - Multi-platform chatbot with RAG and plugins
- Discord-Ollama Chat Bot - TypeScript Discord bot
- Ollama Telegram Bot - Telegram bot
- LLM Telegram Bot - Telegram bot for roleplay
Terminal & CLI
- aichat - All-in-one LLM CLI with Shell Assistant, RAG, and AI tools
- oterm - Terminal client for Ollama
- gollama - Go-based model manager for Ollama
- tlm - Local shell copilot
- tenere - TUI for LLMs
- ParLlama - TUI for Ollama
- llm-ollama - Plugin for Datasette's LLM CLI
- ShellOracle - Shell command suggestions
- LLM-X - Progressive web app for LLMs
- cmdh - Natural language to shell commands
- VT - Minimal multimodal AI chat app
Productivity & Apps
- AppFlowy - AI collaborative workspace, self-hostable Notion alternative
- Screenpipe - 24/7 screen and mic recording with AI-powered search
- Vibe - Transcribe and analyze meetings
- Page Assist - Chrome extension for AI-powered browsing
- NativeMind - Private, on-device browser AI assistant
- Ollama Fortress - Security proxy for Ollama
- 1Panel - Web-based Linux server management
- Writeopia - Text editor with Ollama integration
- QA-Pilot - GitHub code repository understanding
- Raycast extension - Ollama in Raycast
- Painting Droid - Painting app with AI integrations
- Serene Pub - AI roleplaying app
- Mayan EDMS - Document management with Ollama workflows
- TagSpaces - File management with AI tagging
Observability & Monitoring
- Opik - Debug, evaluate, and monitor LLM applications
- OpenLIT - OpenTelemetry-native monitoring for Ollama and GPUs
- Lunary - LLM observability with analytics and PII masking
- Langfuse - Open source LLM observability
- HoneyHive - AI observability and evaluation for agents
- MLflow Tracing - Open source LLM observability
Database & Embeddings
- pgai - PostgreSQL as a vector database (guide)
- MindsDB - Connect Ollama with 200+ data platforms
- chromem-go - Embeddable vector database for Go (example)
- Kangaroo - AI-powered SQL client
Infrastructure & Deployment
Cloud
- Google Cloud
- Fly.io
- Koyeb
- Harbor - Containerized LLM toolkit with Ollama as default backend