Chroma

Chroma is an open-source AI-native embedding database that gives applications a simple four-function API — create a collection, add documents, query, and get — while handling tokenization, embedding generation, and vector indexing automatically behind the scenes. It runs equally well in-memory for prototyping, embedded/persistent on a single machine, or as a standalone server in client-server mode, with official clients for Python and JavaScript.

Core Chroma is written in Rust for performance, with a Python surface layer that exposes collections, metadata filtering, and full-text where_document search. It has become one of the most widely adopted vector stores for retrieval-augmented generation (RAG) pipelines and AI agent memory, and is also offered as a managed service (Chroma Cloud) for teams that don’t want to operate the open-source engine themselves.

What You Get

• A four-function core API: create_collection, add, query, and get, with update/delete also supported
• Automatic embedding generation via built-in embedding functions, or bring-your-own precomputed vectors
• Metadata filtering (where) and full-text document filtering (where_document) combined with vector search
• Three deployment modes: in-memory, persistent single-node, or client-server via chroma run
• Official Python and JavaScript/TypeScript client libraries with a shared HTTP API

Common Use Cases

• Powering retrieval-augmented generation (RAG) pipelines that fetch relevant context for an LLM prompt
• Giving AI agents long-term memory by storing and semantically searching past interactions
• Building internal semantic search over documents, support tickets, or knowledge bases
• Prototyping recommendation or similarity-search features without standing up a dedicated vector DB cluster

Under The Hood

Architecture: Chroma is a polyglot monorepo — the storage/query engine lives under rust/ (blockstore, cache, distance, execution, and index modules) and is exposed to Python through native bindings (chromadb_rust_bindings.pyi), while the chromadb/ Python package layers a friendlier API on top: api/client.py and api/async_client.py expose synchronous and async Client/AdminClient classes, config.py centralizes settings (tenants, databases, auth), and api/models/Collection.py implements the collection abstraction users call add/query/get against. A cli/ module and Docker/Kubernetes deployment assets (k8s/, docker-compose*.yml) support running Chroma as a standalone server rather than purely embedded.

Tech Stack: Core storage and indexing is Rust (69% of the codebase), with Python (16%) as the primary client language and TypeScript (7%) for the JS client; dependencies include pydantic v2 for schema validation, grpcio for internal service communication, opentelemetry for tracing, onnxruntime/tokenizers for the default embedding functions, and typer for the CLI. The project builds with Cargo for the Rust core alongside a standard pyproject.toml for the Python package.

Code Quality: Tests live under chromadb/test/ for the Python layer and within each Rust crate, with bin/test.py and bin/test-remote scripts orchestrating end-to-end and remote-mode test runs; mypy strict-mode config and a py.typed marker indicate the Python surface is fully typed. Given the scale (4,500+ commits, 189 contributors, weekly release cadence per the README), the codebase carries mature CI/CD conventions including bandit security scanning (bandit.yaml) and a documented release process (RELEASE_PROCESS.md).

API Design: The public API is intentionally reduced to four core operations (create_collection, add, query, get) as the README emphasizes, hiding tokenization/embedding/indexing complexity behind sensible defaults while still allowing power users to supply custom embedding functions, precomputed vectors, or combined metadata/full-text filters. This ‘simple by default, flexible when needed’ design, paired with near-identical Python and JS client shapes, keeps the learning curve low for a system solving an otherwise complex problem (approximate nearest-neighbor search at scale).