deepmerge

deepmerge is a small, typed Python library for recursively merging nested data structures — dictionaries, lists, and sets — without writing custom recursive merge logic by hand. It ships three ready-made mergers (always_merger, merge_or_raise, conservative_merger) that cover the most common merge policies out of the box, plus a Merger class for building fully custom strategies keyed by type.

Under the hood, each type (list, dict, set) has its own pluggable strategy list — append, prepend, override, merge, union, use_existing — and a separate fallback/type-conflict strategy resolves cases where merging isn’t well-defined. Strategies are simple functions that return a sentinel (STRATEGY_END) when they can’t handle a value, letting multiple strategies be chained. This makes deepmerge a common building block for merging cascades of YAML/JSON configuration files, environment overrides, and default-plus-user-supplied settings.

What You Get

• Three ready-made mergers — always_merger, merge_or_raise, conservative_merger — covering “always resolve”, “raise on conflict”, and “keep existing” merge policies out of the box
• A Merger class for defining fully custom per-type strategy chains for dicts, lists, sets, or any custom type
• Built-in strategies for dicts (merge, override), lists (append, prepend, override, append_unique), and sets (union)
• Explicit exception types (StrategyNotFound, InvalidMerge) that carry the failing path, base, and next value for diagnosing unmergeable structures
• Full type hints and a py.typed marker for static type checking in downstream projects

Common Use Cases

• Merging a base configuration dict with environment- or user-supplied overrides, such as layered YAML/JSON config files
• Combining multiple partial dictionaries via functools.reduce into one final settings object
• Building CLI tools or frameworks that need predictable, customizable merge semantics for nested option dicts
• Implementing config-cascade patterns where defaults, environment, and per-request overrides must be merged in order

Under The Hood

Architecture The Merger class in merger.py orchestrates merging by dispatching on the runtime type of the base/next values against a list of registered (type, StrategyList) pairs; each type-specific strategy set (DictStrategies, ListStrategies, SetStrategies in the strategy/ package) subclasses a shared StrategyList base (strategy/core.py) that resolves named strategies (e.g. “merge”, “append”) to static methods or accepts plain callables, chaining through them until one returns a real value instead of the STRATEGY_END sentinel. When base and next share no registered type strategy but are structurally related, a FallbackStrategies chain runs; when types are unrelated, TypeConflictStrategies runs. Concrete strategies like DictStrategies.strategy_merge recurse back into config.value_strategy() with an extended path list, giving deep, path-aware merging without a separate tree-walking layer. Swapping the STRATEGY_END sentinel semantics or the StrategyList resolution order would break every built-in and custom strategy, but adding a new type strategy only requires subclassing StrategyList and registering it (or passing a bare callable) — a clean, low-ceremony chain-of-responsibility design for a package this size.

Tech Stack deepmerge is a pure Python 3.8+ package with zero runtime dependencies beyond a conditional typing_extensions backport for Python <=3.9. It’s built with setuptools (>=69) and setuptools_scm for git-tag-derived versioning, written into deepmerge/_version.py at build time. The dev toolchain (declared as an optional ‘dev’ extra) covers black for formatting, mypy for static type checking against the shipped py.typed marker, pytest for tests, and sphinx/sphinx-rtd-theme for documentation hosted on Read the Docs; a GitHub Actions workflow runs the package’s test suite on each push, and a Makefile wires these into build/format/lint/test/docs targets.

Code Quality Tests live under deepmerge/tests/ (test_full.py, test_merger.py, and a strategy/ subpackage with per-strategy test modules) using pytest with parametrized cases, and they exercise the three built-in mergers, custom Merger construction with user-supplied strategy functions, exception paths (asserting InvalidMerge’s path/base/nxt attributes), dict subclass compatibility (OrderedDict, defaultdict), and functools.reduce usage patterns including the destructive-merge gotcha. Type hints are used consistently throughout (from __future__ import annotations, TypeVar generics, a TypeAlias for the strategy callable signature), and the lint target runs validate-pyproject, black —check, and mypy. Naming is consistent — every strategy method follows a strategy_<name> convention resolved by string lookup, and each StrategyList subclass declares a NAME used in error messages.

API Design The public surface is deliberately minimal: three importable mergers (always_merger, merge_or_raise, conservative_merger) solve the common cases with zero configuration, while the Merger class exposes a declarative, data-first API — a list of (type, strategy) tuples — rather than requiring subclassing to customize behavior. Strategies can be named by string for built-in cases or supplied as plain functions matching a documented (config, path, base, nxt) signature, keeping the extension surface identical for built-in and user-defined behavior. The docs (guide.rst, strategies.rst) walk through the destructive-merge caveat and functools.reduce patterns up front, lowering the barrier for anyone moving beyond the three canonical mergers.