Onyx

Onyx is a full-featured, open-source AI platform that acts as the application layer between your organization and large language models. It goes far beyond a simple chat interface—bundling agentic retrieval-augmented generation, a multi-step deep research engine, sandboxed code execution, web search, image generation, voice mode, and artifact creation into a single self-hosted deployment.

At its core, Onyx ships with more than 50 indexing connectors that continuously sync content from sources like Confluence, Notion, Google Drive, Slack, GitHub, Jira, Salesforce, Zendesk, and dozens more. Synced documents are stored in a hybrid vector-plus-keyword index powered by Vespa or OpenSearch, enabling precise semantic and keyword retrieval at enterprise scale.

Custom agents—called Personas—can be configured with specific instructions, curated knowledge sources, and external tool access via MCP (Model Context Protocol) or built-in tools. This lets teams build purpose-built AI assistants without writing code. The platform supports every major LLM provider through LiteLLM, including Anthropic, OpenAI, Gemini, and self-hosted options like Ollama and vLLM.

Onyx ships in two editions: a Community Edition under MIT for core chat, RAG, and agent functionality; and an Enterprise Edition with SSO, RBAC, SCIM provisioning, audit logging, analytics, and whitelabeling. Both editions deploy via Docker Compose or Kubernetes Helm charts in minutes.

What You Get

• Hybrid RAG index — Vespa or OpenSearch-backed vector-plus-keyword search that continuously syncs from 50+ connectors including Slack, Confluence, Notion, Google Drive, GitHub, Jira, and Salesforce
• Agentic deep research — multi-step orchestration loop that plans, searches, synthesizes, and produces cited long-form reports, ranked top of the Onyx deep research benchmark as of early 2026
• Custom Personas (agents) — define AI assistants with unique system prompts, restricted knowledge sources, and tool access via MCP or built-in integrations, shareable across your organization
• Sandboxed code execution — Python interpreter running in an isolated container, used for data analysis, chart rendering, and file manipulation directly inside chat
• Multi-modal capabilities — voice mode with STT/TTS, image generation from prompts, and artifact output (downloadable documents, graphics) from a single conversation
• Universal LLM support — connect to any provider through LiteLLM: Anthropic, OpenAI, Gemini, Azure, or self-hosted models via Ollama, vLLM, or LiteLLM proxy
• Flexible deployment — Docker Compose (standard and lite), Kubernetes Helm charts, and Terraform modules for AWS, GCP, and Azure; Onyx Lite runs under 1 GB RAM for quick evaluation
• Web search with multiple backends — Serper, Google PSE, Brave, SearXNG, in-house crawler, Firecrawl, and Exa integrations for real-time web grounding

Common Use Cases

• Enterprise knowledge assistant — index your company’s Confluence, Notion, and Slack workspaces so employees can ask questions and get cited answers without switching tabs
• Internal developer copilot — connect GitHub, GitLab, and Jira connectors so engineering teams can query codebases, issue trackers, and documentation in a unified AI interface
• Deep research for analysts — use the multi-step research agent to produce comprehensive reports on competitors, markets, or technical topics from both internal and web sources
• Custom department agents — deploy purpose-built Personas for HR, legal, or support teams, each with access only to their relevant data sources and constrained by tailored prompts
• Self-hosted ChatGPT alternative — replace SaaS AI subscriptions with a fully controlled deployment that keeps all conversation data and retrieved content on-premises
• Data analysis automation — upload CSV or database exports and use sandboxed code execution to run analysis, plot charts, and generate summaries within the chat interface

Under The Hood

Architecture Onyx is structured as a layered, service-oriented system divided between an API backend, a model inference server, a Next.js frontend, and a set of background worker processes. The backend is a FastAPI application that handles authentication, chat message processing, connector orchestration, and document indexing through clearly separated modules. Message processing runs a tool-use agentic loop where the LLM emits tool calls, the backend executes search, code, web, or MCP tools, and results are streamed back in a packet-based protocol. The deep research subsystem adds a dedicated orchestrator loop on top, managing multi-step planning, sub-agent dispatching, and final report synthesis as discrete phases. This separation of concerns—chat loop, research loop, indexing pipeline, and connector workers—means each component can be scaled and reasoned about independently without tangling concerns.

Tech Stack The backend is written in Python 3.13 with FastAPI and SQLAlchemy, backed by PostgreSQL as the relational store and either Vespa or OpenSearch as the hybrid vector-and-keyword document index. Redis serves as both a cache and task queue coordinator, while MinIO provides S3-compatible blob storage for files and embeddings. LLM routing is handled by LiteLLM, giving Onyx universal compatibility with Anthropic, OpenAI, Gemini, Azure, and self-hosted providers like Ollama and vLLM. The frontend is a Next.js 14 application using TypeScript. Connector sync workers run as separate background containers and communicate via database state and Redis queues. Deployment targets include Docker Compose (with a minimal Lite profile and a full Standard profile), Kubernetes Helm charts, and Terraform modules for major cloud providers. Observability is covered by Prometheus metrics, Sentry error tracking, and structured JSON logging.

Code Quality The codebase has a comprehensive test suite with 947 test files spanning unit, integration, regression, and daily test categories, alongside a dedicated load testing suite. Python type annotations are used throughout the backend, enforced by mypy with a strict configuration. Pydantic v2 models gate API boundaries and internal data contracts, and FastAPI’s automatic request validation is exploited at every endpoint. Error handling is explicit—custom exception types, structured logging at each failure point, and an indexing pipeline that tracks per-document failure reasons rather than failing entire batches. CI runs linting via ruff and type checking via mypy. The project also ships CLAUDE.md and AGENTS.md files at repo and sub-package level, reflecting an explicit culture of AI-agent-friendly documentation and structured contributor guidance.

What Makes It Unique What distinguishes Onyx from other open-source chat or RAG frameworks is the depth and integration of its full pipeline: most alternatives require assembling connectors, vector stores, and LLM wrappers as separate libraries, whereas Onyx ships all 50+ connector integrations, the hybrid Vespa/OpenSearch index, the agentic chat loop, the deep research orchestrator, sandboxed code execution, and the enterprise access control layer in a single coherent codebase deployable with a single command. The deep research engine specifically claimed a leaderboard position on the Onyx deep research benchmark as of early 2026, reflecting deliberate investment in multi-step reasoning quality rather than relying on the underlying LLM alone. The Lite vs Standard deployment split also shows product maturity—users get a sub-1 GB RAM deployment path for evaluation, graduating to a full distributed stack with Redis, MinIO, and dedicated model servers when ready.

Self-Hosting

Onyx uses a split-license model: all code outside the ee/ directories is released under the MIT License, which permits commercial use, modification, redistribution, and private deployment without any copyleft obligations. The ee/ directories—covering SSO, RBAC, SCIM, analytics, audit logs, whitelabeling, and multi-tenancy—are licensed under the proprietary Onyx Enterprise License, which restricts use without a paid agreement. For most self-hosters, the Community Edition’s MIT core covers everything needed for chat, RAG, custom agents, and all 50+ connectors.

Running the full Standard deployment requires coordinating multiple containers: the FastAPI backend, a Next.js frontend, PostgreSQL, a vector index (Vespa or OpenSearch), Redis, MinIO, an inference model server, and background connector-sync workers. The Docker Compose standard profile and Helm charts handle this orchestration, but the operational burden is real—you own database backups, index maintenance, connector credential rotation, and version upgrades. The Lite deployment profile reduces this significantly by dropping the vector index and background workers, making it viable on a single modest VPS for small teams.

Compared to Onyx Cloud, self-hosters miss out on managed infrastructure, automatic version upgrades, 24/7 support SLAs, and the hosted deep research benchmark features. Enterprise Edition features—SSO via OIDC/SAML, SCIM user provisioning, RBAC, query history auditing, team-level analytics, and custom whitelabeling—require a commercial Enterprise license agreement with the Onyx team. Teams evaluating the self-hosted path should budget for at least two CPU cores, 4 GB RAM minimum (16+ GB for production with the full stack), and S3-compatible storage for document artifacts.