UMAP

UMAP (Uniform Manifold Approximation and Projection) is a Python library for dimension reduction, distributed under the PyPI name umap-learn. It models high-dimensional data as a fuzzy topological structure on a Riemannian manifold and searches for a low-dimensional embedding with the closest possible equivalent structure, giving it a mathematically grounded alternative to t-SNE for both visualization and general-purpose dimensionality reduction.

The library is implemented as a drop-in scikit-learn estimator: umap.UMAP() exposes the familiar fit, transform, and fit_transform API, so it slots directly into existing scikit-learn pipelines. Performance-critical routines (nearest-neighbor descent via the companion pynndescent package, and the embedding optimization loop) are JIT-compiled with Numba rather than hand-written in Cython, which keeps the codebase in pure Python while still scaling to large and high-dimensional datasets, including sparse matrices with over a million dimensions.

Beyond the base algorithm, the package ships several extensions: supervised and semi-supervised dimension reduction via a target array, densMAP for density-preserving embeddings, AlignedUMAP for producing a sequence of related embeddings (e.g. across time slices), and an optional ParametricUMAP built on TensorFlow for learning a parametric embedding function. A umap.plot submodule adds interactive and static visualization helpers layered on matplotlib, datashader, bokeh, and holoviews.

UMAP is widely used in single-cell genomics, general machine-learning preprocessing pipelines, and exploratory data visualization, and is commonly paired with density-based clustering libraries such as HDBSCAN downstream of the embedding step.

What You Get

• A scikit-learn-compatible UMAP estimator with fit / transform / fit_transform, usable as a drop-in replacement for t-SNE or PCA in existing pipelines
• Support for a wide range of distance metrics (Euclidean, cosine, correlation, hamming, jaccard, and more), including user-supplied Numba-JIT’d custom metrics
• Supervised and semi-supervised embedding modes via a target array, plus densMAP for density-preserving projections
• AlignedUMAP for producing a sequence of mutually-aligned embeddings across related datasets or time slices
• Optional ParametricUMAP (requires TensorFlow) for learning a reusable neural-network embedding function instead of a fixed transductive embedding
• A umap.plot submodule with interactive and static plotting helpers built on matplotlib, datashader, bokeh, and holoviews
• Support for sparse-matrix input and precomputed k-nearest-neighbor data to skip redundant computation

Common Use Cases

• Visualizing high-dimensional datasets (image embeddings, gene-expression matrices, word/document embeddings) in 2D or 3D scatter plots
• Dimension reduction as a preprocessing step before downstream machine-learning models or clustering algorithms
• Single-cell genomics workflows, where UMAP is a standard step for visualizing cell-population structure
• Pairing with HDBSCAN or similar density-based clustering on the reduced embedding to find clusters that aren’t well separated in the original feature space
• Semi-supervised or supervised dimension reduction where partial label information should shape the embedding

Under The Hood

Architecture The library centers on umap.UMAP (umap/umap_.py), a scikit-learn BaseEstimator whose fit/fit_transform/transform methods drive a three-stage pipeline: approximate nearest-neighbor search via the companion pynndescent package, construction of a fuzzy simplicial-set graph representing the high-dimensional topology, and optimization of a low-dimensional embedding against that graph using a stochastic-gradient-descent-style layout routine in umap/layouts.py. Initialization of the embedding (spectral, PCA-based, or random) lives in umap/spectral.py, distance-metric implementations in umap/distances.py and umap/sparse.py for the sparse-matrix path, and extensions are implemented as separate estimator subclasses/modules layered on the same core — umap/aligned_umap.py for AlignedUMAP and umap/parametric_umap.py for the optional TensorFlow-backed ParametricUMAP — so the core graph-construction-and-layout separation of concerns is reused rather than duplicated across variants.

Tech Stack Pure Python with Numba (@numba.njit) used throughout the hot paths (distance functions, layout optimization, sparse-matrix helpers) for near-C performance without a Cython build step. Core dependencies are scikit-learn (base estimator API and PCA/TruncatedSVD/SpectralEmbedding utilities), scipy (sparse matrices, curve fitting), numpy, tqdm for progress bars, and the author’s own pynndescent package for approximate nearest-neighbor search. Optional extras add matplotlib/datashader/bokeh/holoviews for the plot submodule and TensorFlow for ParametricUMAP. Packaging is standard PEP 621 (pyproject.toml with a setuptools backend); CI runs across Azure Pipelines, Travis, and AppVeyor for cross-platform coverage.

Code Quality The umap/tests/ directory contains an extensive pytest suite covering nearest-neighbor computation, trustworthiness metrics, sparse and dense data input, densMAP, aligned and parametric variants, gradient computation, and edge cases like repeated data points and small toy datasets (e.g. iris). Coverage is tracked via Coveralls and enforced in CI. The main estimator follows scikit-learn conventions for parameter validation (_validate_parameters) and naming, though as a scientific/numerical codebase it favors explicit numeric parameters and docstring-based documentation over strict static typing.

What Makes It Unique UMAP’s differentiator is its theoretical grounding: rather than being a heuristic like t-SNE, it derives the embedding objective from a fuzzy-topological-set formalism, which in practice gives faster runtimes and better preservation of global structure alongside local neighborhoods. The library extends that single foundation into several distinct capabilities — supervised/semi-supervised reduction, density-aware embedding (densMAP), sequences of aligned embeddings, and a learnable parametric variant — from one shared graph-construction core, rather than shipping them as unrelated bolt-ons.