turbovec

turbovec is a Rust vector index with Python bindings built on Google Research’s TurboQuant algorithm, a data-oblivious quantizer that compresses high-dimensional embeddings to 2-4 bits per coordinate with near-optimal distortion and no training phase. It ships hand-written SIMD search kernels (NEON on ARM, AVX-512 VNNI and AVX2 on x86) that outperform FAISS’s IndexPQFastScan on both search speed and memory footprint across measured configurations.

Beyond raw quantization, it supports online ingest with no rebuild step, crash-safe incremental saves via sync(), search-time id/bitmask filtering that short-circuits unneeded SIMD work, and an optional per-coordinate calibration pass (TQ+) that recovers recall on embeddings that drift from the algorithm’s asymptotic distribution assumptions. Drop-in adapters exist for LangChain, LlamaIndex, Haystack, and Agno, making it a practical local, privacy-preserving vector store for RAG pipelines.

What You Get

• A TurboQuantIndex / IdMapIndex API in both Rust and Python with no training step required before adding vectors
• SIMD-accelerated search kernels (NEON, AVX-512, AVX2, scalar fallback) selected at runtime via feature detection
• Crash-safe incremental persistence (sync) plus whole-file write/load snapshots in a versioned .tv/.tvim format
• Search-time id-allowlist and bitmask filtering evaluated inside the SIMD kernel at block granularity
• Optional TQ+ per-coordinate calibration that improves recall on embeddings whose coordinate distribution drifts from the theoretical assumption
• Ready-made integrations for LangChain, LlamaIndex, Haystack, and Agno vector store interfaces

Common Use Cases

• Running a fully local, privacy-preserving vector store for a RAG pipeline without a managed service
• Shrinking a large embedding corpus (e.g. 10M documents at 31GB float32) down to a few GB via 2-4 bit quantization
• Serving filtered dense retrieval where a candidate set is pre-narrowed by SQL/BM25/ACL before a vector rerank
• Maintaining a continuously growing, mutable index that supports online adds and O(1) removes without periodic retraining
• Swapping in a faster, smaller in-memory vector store behind an existing LangChain/LlamaIndex/Haystack/Agno pipeline

Under The Hood

Architecture A TurboQuantIndex in lib.rs orchestrates a pipeline of small, single-purpose modules: rotation.rs applies the block-Hadamard random rotation, codebook.rs derives the Lloyd-Max centroids for the target bit width, encode.rs handles quantization and the optional TQ+ per-coordinate calibration, pack.rs bit-packs codes into the SIMD-blocked layout, and search.rs holds the platform-specific scoring kernels. id_map.rs wraps the core index to expose stable external ids over swap-and-pop removal, and io.rs/io_v7.rs implement versioned, crash-safe serialization including incremental sync(). The module docs state an explicit, verifiable invariant — every cache reachable through &self describes exactly the rows the index currently holds — and both the blocked layout and packed rows are lazily materialized via OnceLock so concurrent readers never block on cache population.

Tech Stack The project is a two-crate Cargo workspace: turbovec (the core Rust library, MSRV 1.89 for stabilized AVX-512 intrinsics) and turbovec-python (a cdylib built with pyo3 0.29 and numpy 0.29, packaged via maturin). rayon 1.12 drives parallel add/search; rand/rand_chacha and statrs are pinned to exact versions because their output bytes are baked into the on-disk quantization format, with each pin’s rationale documented inline. x86_64 builds target the x86-64-v2 baseline while AVX-512/AVX2 kernels are gated behind is_x86_feature_detected! for runtime dispatch, and NEON kernels cover ARM.

Code Quality The turbovec/tests/ directory holds over 35 integration test files covering adversarial durability and fuzzing, concurrent search safety, calibration bounds, cross-platform kernel correctness (including a dedicated AVX-512 tail-case simulation), and byte-for-byte format version compatibility. CI (.github/workflows/ci.yml) is backed by a dedicated mutation-testing workflow (mutants.yml), a cargo-deny-based supply-chain check (deny.toml, supply-chain.yml), and a security-audit workflow, plus an MSRV job that cross-verifies the declared Rust floor against both x86_64 and aarch64 targets with documented reasoning for why an aarch64-only check would under-report it.

API Design The public surface is intentionally small — new, add, search, write, load, sync — mirrored consistently between the Rust and Python bindings, with IdMapIndex adding only add_with_ids and remove on top of the same shape. Errors are typed (AddError, CalibrateError, ConstructError, FromPartsError, SearchError) rather than surfaced as panics, docs/api.md gives a full reference, and drop-in adapters for four popular RAG frameworks reduce adoption to a single import swap.