sentence-transformers

Sentence Transformers (also known as SBERT) is a Python framework built on PyTorch and Hugging Face Transformers for computing dense embeddings and reranking scores from text and other modalities. It exposes a small set of model classes — SentenceTransformer, CrossEncoder, SparseEncoder, and MultiVectorEncoder — each wrapping a different retrieval paradigm behind a nearly identical encode()/similarity()/rank() API, so switching between bi-encoder, cross-encoder, sparse, and late-interaction (ColBERT-style) retrieval is a matter of swapping the class, not relearning the library.

The project maintains direct access to over 15,000 pretrained models hosted on the Hugging Face Hub, spanning general-purpose embeddings, domain-specific rerankers, and multilingual models drawn from the MTEB leaderboard, alongside a full training stack — 20+ loss functions for embedding models, additional losses for rerankers and sparse encoders, and a trainer built around the Hugging Face Trainer conventions — for teams that need to fine-tune rather than consume off-the-shelf checkpoints. It is the de facto standard dependency behind most Python semantic-search, RAG, clustering, and paraphrase-mining pipelines.

What You Get

• A SentenceTransformer class for computing dense embeddings from text, with .encode() and .similarity() covering the common bi-encoder workflow in two calls
• A CrossEncoder class for reranking (query, passage) pairs with .predict() and .rank(), used as the second stage of retrieve-and-rerank pipelines
• A SparseEncoder class producing SPLADE-style sparse embeddings, plus sparsity statistics via SparseEncoder.sparsity()
• A MultiVectorEncoder class for ColBERT-style late-interaction retrieval, encoding queries and documents into per-token vector sequences scored with MaxSim
• Direct integration with 15,000+ pretrained models on the Hugging Face Hub, loaded by name with no separate download step
• A full training stack (trainer.py, fit_mixin.py) with 20+ embedding losses and dedicated losses for rerankers and sparse encoders, for fine-tuning custom models
• Embedding quantization utilities (quantize_embeddings) supporting int8, uint8, and binary precisions to cut storage and search costs
• Composable neural network modules (Pooling, Dense, CNN, LSTM, WeightedLayerPooling, StaticEmbedding) for assembling custom encoder architectures from scratch

Common Use Cases

• Building semantic search over a document or product corpus using dense embeddings and cosine similarity
• Adding a reranking stage to a retrieval pipeline with CrossEncoder to improve top-k precision after an initial dense retrieval pass
• Clustering or deduplicating large text collections by embedding similarity instead of exact-match heuristics
• Mining paraphrases or near-duplicate sentences across large corpora
• Powering the retrieval component of a RAG (retrieval-augmented generation) pipeline
• Fine-tuning a domain-specific embedding or reranker model on labeled or weakly-labeled pairs using the built-in trainer and loss functions

Under The Hood

Architecture The library is organized around a BaseModel shared by four public model classes (SentenceTransformer, CrossEncoder, SparseEncoder, MultiVectorEncoder), each living in its own top-level package (sentence_transformer/, cross_encoder/, sparse_encoder/, multi_vector_encoder/) alongside a shared base/ package for common modality and module abstractions. SentenceTransformer itself (in sentence_transformer/model.py) composes multiple inheritance from BaseModel and a separate FitMixin (fit_mixin.py) to keep training logic out of the core inference class, while a standalone trainer.py and training_args.py hold the newer Hugging-Face-Trainer-style fine-tuning path. Model internals are built from composable nn.Module layers under sentence_transformer/modules/ (Pooling, Dense-equivalents, CNN, LSTM, WeightedLayerPooling, StaticEmbedding) chained sequentially, so a full model is just an ordered list of these modules plus a Transformer backbone from base/modules.py — changing the pooling or backbone module is a drop-in swap rather than a rewrite of the encoding pipeline.

Tech Stack Built on PyTorch (torch>=2.2) and Hugging Face transformers (pinned to the 5.x line) with huggingface-hub for model download/caching, tokenizers, scikit-learn, and scipy as core dependencies. Optional extras cover vision/audio/video modalities (delegated to transformers[vision|audio|video]), training (datasets, accelerate), and inference acceleration (optimum-onnx, optimum-intel[openvino], plus faiss-cpu/usearch for approximate nearest-neighbor search in the dev extras). The package builds with setuptools, ships a py.typed marker for type-checker consumers, and uses ruff for linting/import-sorting.

Code Quality The repository ships 109 test files under tests/, mirroring the four model packages plus backend/, base/, and util/ test suites, run with pytest (pytest-cov, pytest-xdist, pytest-subtests) and marker-gated slow/custom test tiers so CI can skip expensive model-download tests by default. Three separate GitHub Actions workflows (tests.yml, quality.yml, min-versions.yml) run the suite, lint with ruff/pre-commit, and verify compatibility against the package’s minimum supported dependency versions. Docstrings throughout model.py and related modules are extensive and Napoleon/Sphinx-formatted, and the codebase is typed enough to opt into static type checking (tool.ty.rules config) with only narrow, documented exceptions.

API Design The standout design choice is a nearly identical encode()/similarity()/rank() surface shared across all four retrieval paradigms (dense, cross-encoder, sparse, multi-vector), so switching retrieval strategy is a class swap rather than a new API to learn. Getting started requires two lines — load a model by Hub name, call .encode() — with prompt templates, quantization, and multi-process encoding available as opt-in kwargs rather than required configuration, keeping the common path minimal while still exposing the full training and customization surface for advanced users.