NoteGen

NoteGen is a cross-platform, open-source note-taking application built around a single principle: capture first, organize later. Instead of forcing you to write a well-formed note the moment an idea strikes, NoteGen lets you save raw material — text snippets, voice recordings, screenshots, images, web links, PDFs, and todos — into a capture inbox, then use AI to turn those fragments into structured Markdown documents when you are ready.

The AI layer is unusually capable for a local-first app. A full agent runtime with 40+ typed tools lets you query your notes in natural language, generate weekly reports from recent records by tag or time range, rewrite and translate content, and chain multi-step operations across the file system and editor. RAG combines vector embeddings with BM25 hybrid retrieval so questions are answered from your own knowledge base, not from generic model training. An MCP server manager — implemented in Rust — handles both stdio and HTTP MCP protocols and can install Node.js or Python interpreters locally to run external AI tools without leaving the app.

All notes are stored as plain Markdown files in a workspace you control. Sync is handled via GitHub, GitLab, Gitee, Gitea, S3, or WebDAV with full conflict detection and resolution. The editor is powered by Tiptap 3 with extensions for KaTeX math, Mermaid diagrams, tables, task lists, and code highlighting. SiliconFlow provides free access to Qwen, BGE-M3, and GLM-4V models out of the box, but any OpenAI-compatible provider works. The result is a capable writing environment that runs entirely on your machine with no subscriptions, no ads, and no cloud lock-in.

What You Get

• Capture inbox with multiple input types - Save text, voice recordings, screenshots with OCR, images with AI recognition, web links with scraped content, PDFs, and todos in a unified record inbox without deciding where they belong upfront.
• AI-driven note generation - Select records, pick a template (or write your own prompt), and have AI transform fragmented captures into a coherent Markdown document with headings, summaries, and logical flow — editable immediately after generation.
• Full agent runtime with 40+ tools - An embedded AI agent can read, write, search, rename, move, and tag files; execute Skills scripts; query the knowledge base; manage memories; and interact with the editor — all via natural language instructions with permission confirmation for destructive actions.
• RAG knowledge base with hybrid retrieval - Vector embeddings (BAAI/bge-m3) plus BM25 keyword search combine for context-aware answers drawn from your own Markdown files, with re-ranking to surface the most relevant passages.
• MCP server management - A Rust-backed MCP manager launches and communicates with external MCP servers over stdio or HTTP, including installing Node.js or Python runtimes locally, so AI tools like web search or code execution work inside the app.
• Custom Skills and templates - Define reusable AI workflows (Skills) following the AgentSkills.io specification — with scripts, reference documents, and assets — that can be triggered by the agent or run manually for repeatable tasks like generating weekly reports.
• Cross-platform sync via six providers - Sync notes through GitHub, GitLab, Gitee, Gitea, WebDAV, or S3 with conflict detection, resolution dialogs, and automatic pull-before-push safety checks.
• Rich Tiptap editor with extensions - A WYSIWYG Markdown editor with KaTeX math rendering, Mermaid diagrams, tables, task lists, syntax-highlighted code blocks, drag-and-drop node reordering, outline panel, search-and-replace, and PDF/image export.
• Configurable multi-model AI routing - Assign different LLM providers and models for chat, embedding, OCR/VLM, audio transcription, and inline editing — so you can use a free model for embeddings and a smarter one for writing.
• Activity heatmap and writing stats - Session-based writing activity is tracked and visualized as a GitHub-style heatmap using AntV, turning daily writing patterns into an at-a-glance overview of your knowledge-building habit.

Common Use Cases

• Meeting notes and quick debriefs - Capture raw talking points and action items during a meeting via text or voice, then use a template to generate a structured summary with decisions, owners, and next steps after the call ends.
• Research and reading notes - Save web links, PDF excerpts, and screenshots while researching a topic; when ready, ask the AI agent to organize all records tagged ‘research-project-x’ into a structured analysis document.
• Weekly reports and work logs - Define a ‘weekly report’ Skill that pulls all records from the past seven days filtered by work tags and generates a formatted report — run it every Friday without rethinking the prompt.
• Personal knowledge base with Q&A - Build a library of Markdown notes over time; use the RAG chat interface to ask questions like ‘what were my conclusions about X?’ and get answers grounded in your own writing.
• Developer documentation - Save code snippets, terminal output, GitHub links, and architecture notes as records; use the agent to generate annotated documentation files with diagrams via Mermaid and code blocks.
• Language learning and translation - Capture foreign-language text, use the inline AI translation tool to annotate it, save both versions to the knowledge base, and later query it by topic to review vocabulary in context.

Under The Hood

Architecture NoteGen uses a layered modular architecture that cleanly separates UI rendering, domain state, business logic, data access, and native system operations. The frontend is organized by domain — each major feature (agent, RAG, sync, skills, MCP, recording) lives in its own library module with typed interfaces, while Zustand stores per domain decouple component rendering from data mutations. The Tauri backend exposes a minimal set of Rust commands (MCP server lifecycle, AI streaming, backup, skills import) and delegates file, SQL, and platform operations to typed Tauri plugins. The agent subsystem is particularly well-structured: a tool registry composes 40+ domain-specific tools, a permission engine gates destructive operations, a recovery manager auto-repairs malformed tool calls, a trace recorder enables debugging, and an event bus broadcasts state changes — all wired together in a single runtime that iterates up to 15 steps before stopping.

Tech Stack Next.js 15 with React 19 runs inside Tauri 2 to produce native desktop binaries for Windows, macOS, and Linux and alpha mobile apps for Android and iOS, all from a shared TypeScript codebase. Tiptap 3 powers the rich-text editor with extensions for KaTeX math, Mermaid diagrams, tables, task lists, and collaborative editing primitives. SQLite via tauri-plugin-sql stores structured records, chat history, memories, and vector documents, while plain Markdown files on disk serve as the canonical note format. The RAG pipeline uses BAAI/bge-m3 for vector embeddings and combines cosine similarity search with a BM25 index for hybrid retrieval with re-ranking. The MCP server manager is written in Rust and handles both JSON-RPC-over-stdio and HTTP-based MCP protocols. Tailwind CSS 4 and Radix UI via shadcn/ui form the design system; DnD Kit handles drag-and-drop; AntV renders activity heatmaps; Framer Motion drives transitions. All AI calls go through the OpenAI-compatible SDK, supporting SiliconFlow, OpenAI, and any compatible provider.

Code Quality Test coverage is limited — only three test files exist, all using Node’s built-in test runner with strict assertions to validate utility functions for vector document key normalization, record view mode persistence, and file paste targeting. The rest of the codebase has no automated tests. TypeScript strict mode is enforced throughout, and the type system is used consistently for tool schemas, agent types, sync payloads, and database interfaces. Inline comments are abundant in library modules (primarily in Chinese), documenting function intent and non-obvious logic. Error handling is functional but generic — errors surface as toast notifications with string messages rather than typed error classes. CI is limited to a release pipeline; there is no test runner or lint check in continuous integration, which means code quality relies primarily on TypeScript and developer discipline.

What Makes It Unique NoteGen’s most distinctive technical choices converge on making AI genuinely operational in a local-first context. The MCP server manager in Rust can download and install Node.js or Python runtimes on demand, then launch and communicate with MCP-compliant tools over stdio or HTTP — meaning users can add web search, code execution, or any MCP-compatible capability without external setup. The agent runtime’s permission engine distinguishes between read-only and mutating tool calls and surfaces a confirmation dialog for the latter, making autonomous file operations safe enough for everyday use. The Skills system follows the community AgentSkills.io specification, enabling shareable, versioned AI workflow packages complete with scripts, reference documents, and keyword matching for automatic agent invocation. Finally, the RAG implementation combines dense vector retrieval with BM25 sparse retrieval and a re-ranking pass, a technique typically found in enterprise search systems, applied here to a personal knowledge base running entirely on a laptop.

Self-Hosting

NoteGen is released under the GNU General Public License v3.0, which means you are free to use, modify, and distribute the software, but any derivative work you distribute must also be released under GPL-3.0. For personal and internal use this poses no restriction — you can run and modify NoteGen freely without releasing anything. However, if you intend to ship a modified version to third parties or embed it in a commercial product, you must open-source the combined work under the same license. The copyright holder is codexu, so there is no dual-licensing arrangement or commercial exception available at the time of writing.

Running NoteGen yourself is straightforward from the user’s perspective — you download a pre-built binary and point it at a workspace folder. There is no server to operate; all data stays on the device. The operational burden for individuals and small teams is minimal. The Tauri-based architecture means updates are distributed as binary releases (around 20 MB), and the app can self-update via tauri-plugin-updater. Sync is delegated to external services (GitHub, S3, WebDAV) that you already manage, so there is no NoteGen-specific infrastructure to maintain. The only ongoing dependency is the AI provider — SiliconFlow provides free model access out of the box, but you can configure any OpenAI-compatible API endpoint.

There is currently no managed cloud version, hosted tier, or paid enterprise offering from the project. Everything available in the app is available to self-hosters. The trade-off compared to commercial note-taking tools is the absence of live collaboration, browser-based access, granular permission management, and professional support channels. Mobile apps (Android and iOS) are in alpha and require TestFlight for iOS. If your team needs multi-user real-time editing, role-based access control, or guaranteed SLAs, NoteGen is not designed for that use case — it is built as a personal knowledge management tool for individuals and solo developers.