ollama-ai-provider-v2

ollama-ai-provider-v2 is a Vercel AI SDK provider that talks directly to a local or remote Ollama server, letting you swap in self-hosted models (Llama, Mistral, DeepSeek, Qwen, and more) anywhere the AI SDK’s generateText, streamText, or embed functions are used. It implements Ollama’s native REST API rather than an OpenAI-compatible shim, so it can expose features the shim drops: chain-of-thought “thinking mode” for reasoning models, native Ollama sampling parameters like num_ctx and repeat_penalty, and both chat and legacy completion endpoints.

It targets AI SDK v7’s provider specification (LanguageModelV4/EmbeddingModelV4), ships with only two runtime dependencies, and is built to run identically in Node.js, browsers, and edge runtimes. Tool calling, multi-step generation, and embedding batches are all supported, along with a createOllama() factory for pointing at custom or remote Ollama-compatible servers.

What You Get

• A drop-in AI SDK provider (ollama(modelId)) usable anywhere the SDK’s generateText, streamText, or embed functions are used
• Native Ollama request/response handling instead of an OpenAI-compatible shim, preserving Ollama-specific behavior
• Streaming text generation over NDJSON with a dedicated stream processor and finish-reason mapping
• Tool/function calling support with multi-step execution via the AI SDK’s stopWhen/stepCountIs pattern
• A reasoning (“thinking mode”) option for chain-of-thought models like deepseek-r1, exposed both as a top-level AI SDK option and a native providerOptions.ollama.think flag
• Pass-through access to Ollama’s native sampling parameters (seed, num_ctx, repeat_penalty, top_k, min_p) via providerOptions.ollama.options
• Text embedding support, including batch embedding calls, via ollama.embedding(modelId)
• A createOllama() factory for connecting to remote or custom-configured Ollama servers with custom headers and base URLs

Common Use Cases

• Running open-weight local models (Llama, Mistral, Qwen, Gemma) behind the same AI SDK call sites used for hosted providers
• Building privacy-sensitive or offline-capable AI features that must not send data to a third-party API
• Prototyping agent/tool-calling flows against a free local model before switching to a hosted provider
• Using reasoning models like DeepSeek-R1 with explicit chain-of-thought output via thinking mode
• Generating text embeddings locally with models like nomic-embed-text for search or RAG pipelines without external API costs

Under The Hood

Architecture The package is organized as a modular provider with dedicated directories for the chat/responses model (responses/), a legacy completion model (completion/), an embedding model (embedding/), message/prompt adaptors (adaptors/), and shared HTTP/config plumbing (common/). ollama-provider.ts implements a factory, createOllama(), that builds a callable provider object conforming to the AI SDK’s ProviderV4 interface, wiring each model type to a shared OllamaConfig (base URL, headers, fetch override). Classes such as OllamaResponsesLanguageModel implement the LanguageModelV4 interface’s doGenerate/doStream methods by delegating request construction to an OllamaRequestBuilder and response parsing to an OllamaResponseProcessor/OllamaStreamProcessor pair, so adapting to a new Ollama API revision touches the builder/processor layer rather than the model class itself.

Tech Stack Written in TypeScript, built with tsup into dual ESM/CJS output plus type declarations, targeting Node.js 22+. Runtime dependencies are minimal — @ai-sdk/provider and @ai-sdk/provider-utils supply the AI SDK’s model interfaces and HTTP helpers (postJsonToApi, combineHeaders) — with ai and zod as peer dependencies. It is a pure HTTP client over Ollama’s native REST endpoints (/api/chat, /api/generate, /api/embed); there is no database or ORM layer. Tests run under vitest across two separate configs, one for Node and one for an edge runtime (via @edge-runtime/vm), with msw mocking HTTP responses.

Code Quality The repository ships a broad test suite covering the responses processor, stream processor, tool preparation, message conversion, embedding model, completion model, error handling, and finish-reason mapping, with reusable test-helper utilities for constructing mock Ollama server responses. Error handling is explicit and schema-driven: a dedicated error module defines a validated error schema and failed-response handler, and API responses are parsed through Zod schemas rather than trusted as untyped JSON. Naming is consistent and descriptive throughout, TypeScript is used end-to-end against the AI SDK’s typed interfaces, and CI runs the test suite and a lint/build check on every pull request and push to the main branch.

API Design The public surface is deliberately small: a callable ollama(modelId) for the common case, plus .chat(), .completion(), and .embedding() for explicit model selection, and createOllama() for custom server configuration — all mirroring the shape developers already know from other AI SDK providers, so switching from a hosted model to a local Ollama model is close to a one-line change. Ollama-specific capabilities (thinking mode, native sampling options) are layered on through the AI SDK’s existing providerOptions extension point rather than bespoke APIs, keeping the surface consistent while still exposing provider-specific power.