hdbscan

hdbscan is a Python implementation of Hierarchical Density-Based Spatial Clustering of Applications with Noise (HDBSCAN). It runs DBSCAN across a full range of epsilon values and integrates the results to find the clustering that is most stable over that range, which lets it discover clusters of varying density in a single pass rather than requiring a single fixed density threshold like DBSCAN.

The library is built as a drop-in scikit-learn estimator (BaseEstimator, ClusterMixin), so it accepts the same array, DataFrame, sparse-matrix, and precomputed-distance-matrix inputs as other sklearn clusterers and slots directly into existing pipelines. Its performance-critical routines — minimum spanning tree construction, tree condensation, and Boruvka-based nearest-neighbor search — are written in Cython, which the project reports as orders of magnitude faster than the reference Java implementation.

Beyond the core clustering call, the package exposes soft clustering (per-point cluster membership strengths), GLOSH outlier scoring, a flat-clustering module for extracting a fixed number of clusters, prediction utilities for scoring new points against a fitted model, a robust single-linkage clusterer, and a branch-detection module (implementing the FLASC algorithm) for finding sub-cluster branching structure. It is maintained under the scikit-learn-contrib organization and distributed via both PyPI and conda-forge with prebuilt wheels for major platforms.

What You Get

• HDBSCAN estimator - a scikit-learn BaseEstimator/ClusterMixin clusterer with a familiar fit/fit_predict API that accepts arrays, DataFrames, sparse matrices, or precomputed distance matrices.
• Automatic variable-density clustering - finds clusters of differing density in one run by integrating DBSCAN results across a range of epsilon values, instead of requiring one global density threshold.
• Soft clustering and outlier detection - per-point cluster membership strength vectors plus GLOSH outlier scores for identifying noise points without extra computation.
• Robust single linkage clusterer - a high-performance implementation of Chaudhuri and Dasgupta’s robust single-linkage algorithm, with hierarchy plotting and flat-cluster extraction at any cut level.
• Branch detection (FLASC) - a BranchDetector class that finds branching sub-structure within clusters, useful when cluster shape itself is informative.
• Prediction utilities - approximate_predict, membership_vector, and related functions for scoring new, unseen points against a previously fitted clusterer with joblib caching support.

Common Use Cases

• Exploratory data analysis - a data scientist runs HDBSCAN on an unlabeled dataset to discover natural groupings without pre-specifying the number of clusters.
• Anomaly and outlier detection - an engineer uses GLOSH outlier scores from a fitted clusterer to flag unusual data points in sensor or transaction data.
• Geospatial and trajectory clustering - an analyst clusters GPS points or geographic coordinates using haversine or other BallTree-supported metrics to find dense regions of activity.
• Topic or embedding clustering in ML pipelines - a practitioner clusters document or image embeddings (e.g. from UMAP-reduced vectors) since HDBSCAN handles variable-density clusters better than k-means-style approaches.
• Streaming or incremental scoring - a team fits HDBSCAN once on historical data, then uses approximate_predict to assign new incoming points to existing clusters without refitting.

Under The Hood

Architecture The package splits cleanly between a Python-facing API layer and Cython-compiled computational core. hdbscan_.py defines the HDBSCAN class as a scikit-learn BaseEstimator/ClusterMixin, orchestrating input validation (check_array), spatial index selection (KDTree/BallTree from scikit-learn, or a Boruvka MST algorithm for larger metric spaces), and delegating the actual mutual-reachability, minimum-spanning-tree, and tree-condensation work to compiled .pyx modules (_hdbscan_linkage, _hdbscan_tree, _hdbscan_boruvka, _hdbscan_reachability). Separate modules extend the core estimator without bloating it: flat.py for fixed-cluster-count extraction, prediction.py for scoring new points via cached PredictionData, branches.py for the FLASC branch-detection post-processing step, robust_single_linkage_.py for the standalone robust single-linkage variant, and validity.py for cluster validity indices. This layering means the core density-tree algorithm can evolve independently of the sklearn-facing surface.

Tech Stack Built on numpy, scipy, scikit-learn (>=1.6), and joblib for caching and parallelism, with performance-critical code compiled via Cython (3.x) against numpy 2.x C APIs through a custom build_ext command in setup.py. The build system uses setuptools with pyproject.toml-declared build requirements. Distribution covers PyPI wheels and conda-forge packages for macOS, Linux, and Windows, with CI (Azure Pipelines) testing a matrix spanning Python 3.10 through 3.14 across all three platforms.

Code Quality The test suite (hdbscan/tests/) covers the main estimator, flat clustering, branch detection, prediction utilities, and robust single linkage, and includes scikit-learn’s own check_estimator conformance checks to verify the estimator obeys sklearn’s API contract. Tests run via pytest with pytest-cov and pytest-benchmark in CI, with coverage reported to Codecov. No project-wide type-checking or linting configuration (mypy, ruff, flake8) was found in the repository, so type safety relies on runtime validation rather than static enforcement.

What Makes It Unique Unlike flat density-based methods such as DBSCAN, which need a single global density (epsilon) parameter, HDBSCAN integrates across the full range of density thresholds to find the clustering that is most stable overall, letting it recover clusters of genuinely different densities in the same dataset with essentially one intuitive parameter (minimum cluster size). The addition of GLOSH-based outlier scoring, soft membership vectors, and FLASC branch detection gives it a broader analytical toolkit than most clustering libraries, which typically stop at hard cluster labels.