SurfSense

SurfSense is an open-source, privacy-first AI research platform built as a direct answer to the limitations of Google NotebookLM. Where NotebookLM caps sources at 50 per notebook, limits notebooks to 100, and locks you into Google’s LLMs, SurfSense imposes none of those constraints. You can connect unlimited sources, create unlimited search spaces, and choose any LLM—OpenAI, Anthropic, Ollama, vLLM, AWS Bedrock, Azure OpenAI, or any OpenAI-compatible endpoint.

The platform is built around three core pillars: a powerful knowledge ingestion pipeline (27+ connectors including Notion, Slack, Google Drive, GitHub, Gmail, Jira, Discord, Elasticsearch, and a web crawler), a real-time collaborative workspace (shared search spaces with role-based access, live chat, comment threads, and Electric-SQL for multiplayer state sync), and a growing suite of AI output formats (cited reports, two-host podcasts, slide decks, narrated video overviews, and AI images—all generated from your own data).

SurfSense ships a native desktop app (Electron, cross-platform) that brings AI assistance to every application on your computer via Global Assist, Quick Assist for selected text, Screenshot Assist, and a local folder watcher that auto-syncs directories—including Obsidian vaults—into your knowledge base. A no-code automation engine lets you trigger AI agents on a schedule or when documents arrive in a folder, writing results back to Notion, Slack, Linear, or Google Drive.

The backend runs on Python 3.12 with FastAPI, Celery/Redis for async task processing, PostgreSQL with pgvector for hybrid semantic and full-text search, and LangGraph for stateful multi-agent chat workflows. Deployment is a single shell command on Linux/macOS or PowerShell on Windows, with Docker Compose handling all dependencies including Watchtower for automatic daily updates.

What You Get

• Unlimited Sources and Search Spaces - No caps on the number of documents, notebooks, or source files; ingest gigabytes of data across as many search spaces as you need, unlike NotebookLM’s 50-source and 100-notebook limits.
• 100+ LLM Support via LiteLLM - Route requests to OpenAI, Anthropic, Ollama, vLLM, AWS Bedrock, Azure OpenAI, GitHub Models, or any OpenAI-compatible endpoint without rewriting any configuration.
• 27+ External Data Connectors - Sync Notion, Slack, Google Drive, Gmail, GitHub, OneDrive, Dropbox, Jira, Confluence, Discord, Airtable, ClickUp, Elasticsearch, Microsoft Teams, BookStack, Obsidian, MCP servers, and more with periodic auto-sync.
• Real-Time Multiplayer Collaboration - Shared search spaces with live co-editing of chats, comment threads with @mentions, notification inbox, and RBAC with Owner, Editor, and Viewer roles—powered by Electric-SQL for low-latency sync.
• Native Desktop App - Electron app with General Assist (global shortcut), Quick Assist (act on selected text in any app), Screenshot Assist (attach screen regions to chat), and Watch Local Folder for automatic knowledge base sync.
• No-Code Automation Engine - Define scheduled agents (daily briefs, weekly digests), event-triggered automations (fire when a document lands in a folder), and chat-built workflows described in plain English that write results back to Notion, Slack, Linear, or Drive.
• Hybrid Search with Reciprocal Rank Fusion - Combines pgvector semantic search with full-text retrieval, reranking via FlashRank, and time-based filtering so the most recent and relevant chunks surface first.
• Multi-Format Deliverable Studio - Generate cited research reports (PDF, DOCX, LaTeX, EPUB, ODT, HTML), two-host AI podcasts, narrated slide decks and video overviews, and AI images—all from your own knowledge base.
• Browser Extension for Authenticated Web Capture - Save any webpage—including pages behind login—directly into your search spaces from Chrome, Firefox, or Edge with one click.
• Daytona Sandbox for Secure Code Execution - Run agent-generated code in ephemeral, isolated Daytona sandboxes with filesystem access, preventing arbitrary code execution on the host.

Common Use Cases

• Research teams querying proprietary document repositories - A university lab ingests thousands of internal PDFs from Google Drive and Notion, uses a local Ollama model for privacy, and exports cited LaTeX papers from the Deliverable Studio.
• Enterprise knowledge management with write-back - An operations team connects Slack, Jira, and Confluence, runs a scheduled weekly digest agent that posts summaries back to their Slack channel, and lets team members query the corpus in a shared search space.
• Journalists and content creators building multimedia research - A reporter uses the browser extension to save articles, generates a two-host podcast from the collected sources in under 20 seconds, and exports an HTML newsletter report with inline citations.
• Developers using Obsidian as a personal knowledge base - An engineer points Watch Local Folder at their Obsidian vault, enabling semantic search and AI chat across all their notes while keeping the vault local and private.
• Legal and compliance teams on private infrastructure - A law firm self-hosts SurfSense on-premises, connects OneDrive and Dropbox for contract ingestion, and uses RBAC to restrict sensitive search spaces to specific team members.

Under The Hood

Architecture SurfSense has grown from a prototype monolith into a multi-subsystem application with meaningful separation of concerns. The backend is organised around discrete functional domains—an indexing pipeline with incremental chunk reconciliation, a retrieval layer with instrumented hybrid search, a multi-agent chat runtime backed by LangGraph and PostgreSQL checkpointing, a gateway subsystem for streaming ingestion from Discord, WhatsApp, and long-poll sources, a no-code automation engine with typed triggers and a step executor, and an observability layer emitting OpenTelemetry traces and metrics. FastAPI-Users handles authentication with OAuth, SlowAPI enforces rate limiting against Redis, and a Celery worker pool with task-queue isolation handles all async and scheduled work. The codebase still relies on route-level imports rather than dependency injection containers, and service boundaries are occasionally blurry, but the directional improvement over earlier versions is clear and consistent.

Tech Stack The backend is Python 3.12 with FastAPI, fastapi-users for OAuth2 authentication, SQLAlchemy and Alembic for database migrations, and asyncpg for async PostgreSQL access. pgvector stores embeddings for semantic search alongside full-text indexes. Celery with Redis manages background task queues, including isolated queues per task type. LangChain and LangGraph power the multi-agent chat graph with persistent PostgreSQL checkpointing via psycopg3. LiteLLM abstracts 100+ LLM providers. The frontend is Next.js 14 with TypeScript, shadcn/ui components, Jotai for global state, TanStack Query for server state, and Electric-SQL for real-time collaborative sync. The desktop app is Electron, distributed for macOS, Windows, and Linux. Docling, Unstructured, and LlamaCloud handle document ETL across 50+ file formats. Daytona provides sandboxed code execution. OpenTelemetry exports traces and metrics to a configurable OTLP collector.

Code Quality The test suite is extensive, with over 1,700 unit tests spanning connectors, ETL pipeline, indexing pipeline, agents, automations, gateway, event bus, middleware, notifications, observability, and more—organised into clearly separated unit, integration, and e2e layers with shared fixtures and harness utilities. Type hints are comprehensive throughout the Python codebase; Ruff, Bandit, and Detect-Secrets run as pre-commit hooks on the backend, while Biome enforces formatting and linting on the TypeScript frontend. Error handling is explicit in most subsystems, with a custom SurfSenseError hierarchy and structured logging that includes OpenTelemetry trace and span IDs. Some route handlers still catch broad exceptions, but the trend toward typed error contracts is visible in newer code.

What Makes It Unique SurfSense’s most distinctive technical characteristic is its breadth of genuinely integrated features rather than surface-level connectors. The no-code automation engine uses a typed trigger-and-action model with a step executor that can call back into 27+ connected services as write targets, not just read sources. The multi-agent chat uses LangGraph’s stateful graph with per-session PostgreSQL checkpointing, enabling resumable, interruptible agent workflows. Electric-SQL provides real-time CRDT-style sync for collaborative workspaces without a separate WebSocket server. The desktop app’s Watch Local Folder feature syncs directory changes into the knowledge base in the background, making it the only open-source NotebookLM alternative to ship native OS integration alongside the web product. The credit reservation system in premium_quota.py—reserve-finalize-release lifecycle tied to LiteLLM cost reporting—shows genuine multi-tenant cost awareness in the self-host codebase.

Self-Hosting

SurfSense is licensed under the Apache License 2.0, which is a permissive open-source license. You can use it commercially, modify it, distribute it, and sublicense it without any copyleft obligations—meaning your own application does not need to become open-source simply because it uses SurfSense. The desktop app ships under the MIT license. There are no proprietary modules, no open-core enterprise tier gated behind a separate license, and no code obfuscation in the self-hosted distribution.

Running SurfSense yourself requires meaningful infrastructure: PostgreSQL 16+ with the pgvector extension enabled, Redis for Celery task queues and rate-limiting state, object storage (local filesystem, AWS S3, or Azure Blob) for uploaded files and generated media, and sufficient CPU and RAM to run the Python backend, Celery workers, and Next.js frontend concurrently. The install script automates Docker Compose setup and includes Watchtower for daily automatic image updates, which reduces maintenance overhead but requires that you trust upstream image builds. For video and podcast generation you also need FFmpeg installed in the container. Scaling horizontally means running multiple Celery worker replicas—the codebase includes task-queue isolation to prevent job collisions—but you manage load balancing, database connection pooling, and Celery autoscaling yourself.

The hosted cloud version at surfsense.com adds a credit-based billing system (reserve-finalize-release lifecycle tied to LiteLLM provider cost reporting) and manages all infrastructure, automatic upgrades, and SLAs on your behalf. Self-hosters give up managed uptime, auto-scaling, paid support channels, and the convenience of a pre-configured LLM and storage tier. For teams with strict data residency requirements or air-gapped environments, self-hosting is the right choice; for teams that want to start immediately without DevOps overhead, the cloud tier is a practical entry point.