LangChain Rust

LangChain Rust is the Rust implementation of the LangChain paradigm for building applications on top of large language models. It provides composable building blocks - LLM clients, prompt templates, chains, agents, tools, embeddings, and vector stores - so you can assemble retrieval-augmented generation, conversational agents, and multi-step LLM workflows in a strongly typed, memory-safe language. It brings the ergonomics of the Python and JavaScript LangChain ecosystems to Rust’s performance and reliability.

What You Get

• Unified LLM and embedding clients for OpenAI, Azure, Ollama, Anthropic Claude, MistralAI, and FastEmbed
• Chains for conversation, retrieval, sequential steps, Q&A, and SQL generation
• Agents with tool calling, including search, Wolfram, and command-line tools
• Vector store integrations for Postgres, Qdrant, SQLite, OpenSearch, and SurrealDB
• Document loaders and text splitters for building RAG pipelines

Common Use Cases

• Building retrieval-augmented chatbots grounded in your own documents
• Running tool-using agents that call APIs and search engines from Rust services
• Adding LLM features to existing high-performance Rust backends

Under The Hood

Architecture

The crate is organized around trait abstractions in src for LLM, Embedder, VectorStore, Chain, Agent, and Tool, with concrete implementations per provider selected through feature flags. A chain composes a prompt template, an LLM, and optional memory; agents wrap an LLM with a tool registry and an execution loop that parses tool calls and feeds observations back. Document loaders and text splitters feed embeddings into a vector store to support retrieval chains.

Tech Stack

Async Rust built on Tokio and async-trait, with reqwest for provider HTTP calls and serde for request and response modeling. Providers, vector stores, and loaders are gated behind Cargo features to keep builds lean, and roughly forty runnable examples under examples/ exercise each integration.

Code Quality

The codebase spans over 170 source files with modular per-provider layout and 36 files carrying tests, alongside CI, a contributing guide, and Renovate-managed dependencies. Error handling uses typed error enums per module. Coverage is uneven across the many integrations but the core chain and agent paths are exercised by examples and tests.

API Design

The API mirrors the well-known LangChain mental model, so developers coming from Python or JavaScript find familiar concepts - chains, agents, retrievers, and prompt templates. Builder-style constructors and macros reduce boilerplate, and the large example set plus a hosted tutorial site flatten the learning curve for the framework’s breadth.