deepdiff

DeepDiff compares two Python objects of nearly any shape — dicts, lists, sets, tuples, custom classes, NumPy arrays, pandas/polars frames — and reports exactly what changed, including moved list items, type changes, and numeric drift within a tolerance. It ships as a family of tools under one package: DeepDiff for comparisons, DeepHash for content-based hashing of arbitrary objects, DeepSearch/grep for locating values inside nested structures, and Delta for capturing a diff as a standalone, serializable patch that can be replayed against another object to reconstruct it.

The library is built for production use in test suites, data-validation pipelines, and audit/reconciliation systems where naive equality checks (==) are too blunt. It ships with a py.typed marker and full type hints, an optional multiprocessing backend for large ignore_order comparisons, and extensive path/exclusion controls (regex, glob, and exact paths) so callers can tune exactly what is and isn’t considered a difference.

What You Get

• DeepDiff — the core comparison engine: dict/list/set/tuple diffing with order-insensitive and type-change detection modes
• DeepHash — content-based hashing of arbitrary nested objects, usable standalone or as DeepDiff’s internal comparison primitive
• DeepSearch / grep — search for values or patterns anywhere inside a nested object and get back their paths
• Delta — capture a diff as a portable object, serialize it, and apply it to reconstruct the target from the source
• Path-based exclude/include controls — exact paths, regex paths, and (new in 9.1) glob/wildcard paths for exclude_paths/include_paths
• Multiprocessing backend — optional parallel distance computation and subtree diffing for large ignore_order=True comparisons, with deterministic output ordering
• A command-line tool (deep, via the cli extra) for diffing two files without writing Python

Common Use Cases

• Asserting deep equality in test suites with a readable diff instead of a raw assertEqual failure
• Diffing API response payloads or config files across versions/environments to catch unintended drift
• Validating data pipeline outputs by comparing before/after snapshots of records, including NumPy/pandas data
• Building audit trails by capturing a Delta of every change and replaying/reversing it later
• Reconciling two versions of a large nested document (JSON/YAML) while ignoring list-item reordering

Under The Hood

Architecture The core lives in diff.py, where DeepDiff walks two objects recursively and builds a tree of DiffLevel nodes (model.py) via relationship classes chosen by type — DictRelationship, AttributeRelationship, SubscriptableIterableRelationship, SetRelationship, NumpyArrayRelationship. Order-insensitive comparisons fall back to DeepHash-based pairing (deephash.py) to match reordered items by content rather than position. Delta (delta.py) sits on top of the same diff representation to make it replayable, while an internal multiprocessing module isolates the optional parallel backend as pure, picklable module-level functions so it works safely under the spawn start method. Cross-cutting concerns — serialization, distance scoring, colored rendering, caching via LFUCache/DummyLFU — are split into mixins rather than living in the main diff loop, which keeps a genuinely large comparison engine navigable.

Tech Stack Pure Python 3.10+ with a minimal required dependency footprint (cachebox, orderly-set); NumPy, pandas, polars, orjson, PyYAML, click, and pydantic are all opt-in extras rather than hard requirements. Built and packaged with flit_core, dependency management via uv (a committed lockfile is present), and test/lint/type-check tasks orchestrated through nox across the Python 3.10-3.14 matrix in CI.

Code Quality Extensive test coverage — roughly 888 test functions across around 30 files — run with pytest, pytest-benchmark, and pytest-cov, exercised in CI across five Python versions plus PyPy, with coverage reported to Codecov and flake8 enforced. The package ships a py.typed marker and a mypy configuration, signaling an explicit commitment to a type-checkable public API. A dedicated security-focused test file covers the delta-application hardening against attribute-traversal abuse.

API Design The common case is a single DeepDiff(t1, t2) call with sensible defaults, while advanced users get deep configurability — order sensitivity, numeric tolerance, exact/regex/glob path exclusion, and pluggable custom comparison operators — without subclassing. The standout ergonomic choice is Delta: turning a diff into a reversible, serializable patch object is uncommon among comparison libraries, and the newer multiprocessing backend for unordered-collection diffing tackles a known real-world performance pain point while explicitly preserving deterministic output ordering, a tradeoff most similar libraries don’t attempt.