FastEmbed-rs

FastEmbed-rs is a Rust library for generating text and image vector embeddings, sparse embeddings, and reranking scores entirely locally, without calling an external embeddings API. It runs models via ONNX Runtime (through the ort crate) and Hugging Face’s tokenizers crate, supporting a curated set of popular embedding models (BAAI/bge family, and others) downloaded on demand from Hugging Face Hub.

The library is synchronous by default with no dependency on Tokio, making it easy to embed in CLI tools, batch pipelines, or existing sync codebases. It has sibling ports in Python, Go, and JavaScript, all sharing the same model set and naming, which makes it a common choice for teams standardizing local embedding/reranking behavior across a polyglot stack.

What You Get

• Local text embedding generation via curated ONNX models (BAAI/bge-small/base/large, and more)
• Image embedding support for multimodal retrieval pipelines
• Sparse text embedding generation for hybrid dense+sparse search
• Cross-encoder reranking to reorder retrieved documents by relevance
• Automatic model download/caching from Hugging Face Hub with no Tokio/async requirement

Common Use Cases

• Generating embeddings for a RAG (retrieval-augmented generation) pipeline without an external API dependency
• Building a local semantic search index over documents without sending data to a third-party embeddings service
• Reranking a vector-search candidate set to improve relevance before passing results to an LLM
• Running embedding generation inside a Rust CLI tool, batch job, or serverless function with minimal runtime overhead

Under The Hood

Architecture - The crate is organized by embedding modality — text_embedding/, image_embedding/, sparse_text_embedding/, bgem3_embedding/, and reranking/ — each wrapping an ONNX model behind a common init.rs loading path and a shared pooling.rs/similarity.rs for post-processing, so adding a new model type follows an established pattern rather than ad-hoc integration. Tech Stack - Rust 2021 edition, built on ort (ONNX Runtime bindings, pinned to a specific 2.0.0-rc.12), tokenizers for Hugging Face-compatible tokenization, ndarray for tensor math, hf-hub for on-demand model downloads, and optional candle-nn/image features for additional model families — deliberately avoiding Tokio so the library stays usable in synchronous codebases. Code Quality - 7 integration test files exercise the embedding/reranking paths against real downloaded models rather than mocks; the project ships releases via an automated semantic-release GitHub Action (91 releases since October 2023), indicating a disciplined, CI-driven release cadence. API Design - Each embedding type exposes a small, consistent struct/builder pattern (e.g. TextEmbedding::try_new(options)), and the shared model-naming convention across the Python, Go, and JS sibling ports means Rust users switching from another FastEmbed language binding face minimal relearning.