ollama-python

ollama-python is the official Python SDK for Ollama, the local-first LLM runtime. It provides synchronous and asynchronous clients that wrap Ollama’s REST API with typed Pydantic request and response models, covering chat, text generation, embeddings, and full model-lifecycle operations such as pull, push, create, copy, and delete.

Beyond a single local server, the same client classes can target Ollama’s hosted cloud API by swapping the host and adding an API key, so code written against a local model can be pointed at larger cloud-hosted models with a configuration change rather than a rewrite. A built-in helper converts plain Python functions into tool-call schemas automatically, removing a common source of hand-written JSON-schema boilerplate in agentic tool-calling code.

What You Get

• A Client and AsyncClient pair covering chat, generate, embed, and model-management endpoints with matching method signatures
• Typed Pydantic response models (ChatResponse, GenerateResponse, EmbedResponse, ListResponse, etc.) that support both attribute and dict-style access
• Streaming support via stream=True, returning a (async) generator of incrementally parsed response chunks
• Automatic conversion of plain Python functions into Ollama tool-call schemas for agentic/tool-calling workflows
• Built-in Image handling for multimodal chat/generate calls from raw bytes, file paths, or base64 strings
• Hosted web-search and web-fetch helpers (web_search, web_fetch) for agent tool loops

Common Use Cases

• Local LLM prototyping — call ollama.chat() against a locally running model with no cloud API keys or network access required
• Streaming chat UIs — wire AsyncClient().chat(..., stream=True) into a backend SSE/WebSocket endpoint to stream tokens as they’re generated
• Agent tool-calling pipelines — pass plain Python functions to chat(tools=[...]) and rely on automatic schema conversion instead of hand-written JSON schemas
• Cloud-offloaded inference — repoint Client(host='https://ollama.com') with an API key to run larger cloud-hosted models with no code changes beyond configuration
• Model lifecycle automation — script pull, create, and delete calls to manage custom fine-tuned models as part of a deployment pipeline

Under The Hood

Architecture

ollama/_client.py defines a shared BaseClient (header setup, timeout/proxy handling) subclassed by Client (sync, built on httpx.Client) and AsyncClient (async, built on httpx.AsyncClient via anyio), each implementing near-identical method pairs — chat, generate, embed, embeddings, pull, push, create, delete, list, copy, show, ps, web_search, web_fetch — that build request payloads from the Pydantic models in ollama/_types.py, send them through a shared _request/_request_raw helper, and either return a parsed response object or, when stream=True, yield an iterator/async-generator of incrementally parsed JSON chunks; ollama/_utils.py holds a standalone convert_function_to_tool helper invoked from _client.py, and ollama/init.py instantiates one module-level Client and re-exports its bound methods as top-level functions (ollama.chat(), ollama.generate()) for a lower-ceremony scripting layer on top of the class-based API.

Tech Stack

Python (>=3.8) with httpx>=0.27 as the HTTP transport for both clients, pydantic>=2.9 for typed request/response models and JSON-schema generation for tool definitions, and anyio for async primitives; built and versioned via hatchling/hatch-vcs (version derived from git tags), linted and formatted with ruff (custom line-length and quote-style rules), and tested through a hatch-test environment running pytest with pytest-anyio and pytest-httpserver. Distributed on PyPI as the ollama package; its runtime deployment target is simply being pip-installed alongside a locally running or cloud-hosted Ollama server rather than a service of its own.

Code Quality

tests/test_client.py, test_utils.py, and test_type_serialization.py exercise both Client and AsyncClient against pytest-httpserver mocks, covering request construction, streaming, tool-schema conversion, and Pydantic (de)serialization edge cases. Errors are explicit and typed via a custom ResponseError raised from parsed HTTP error bodies rather than generic exceptions; naming stays consistent snake_case mirroring the underlying REST endpoints, and type hints plus Pydantic models give strong type safety throughout _client.py. ruff enforces both linting and formatting, and a GitHub Actions workflow (test.yaml) runs the test suite in CI alongside a separate publish.yaml for releases — comprehensive test coverage, typing, and CI enforcement.

API Design The public API mirrors Ollama’s REST endpoint names almost one-to-one (chat, generate, embed, show, ps), so anyone familiar with Ollama’s HTTP docs needs no translation layer to use the Python client. Getting started requires nothing beyond pip install ollama and a running local server, and the module-level functions let simple scripts skip client instantiation entirely while power users can still drop to Client/AsyncClient for custom hosts, headers, or timeouts. The automatic function-to-tool schema conversion is the most notable ergonomic touch, removing a class of manual JSON-schema boilerplate that many comparable LLM SDKs still require by hand.