lazy-loader

lazy-loader is a small, focused utility from the Scientific Python community that lets library authors replace __init__.py imports with deferred equivalents. Instead of importing every subpackage and function eagerly, lazy_loader.attach swaps in a package’s __getattr__ and __dir__ so submodules and attributes are only imported the first time they’re actually touched — the exact pattern scikit-image and other scientific-python-spec projects use to keep import <package> fast even as the library grows.

Beyond package-internal imports, lazy_loader.load provides the same deferred behavior for external dependencies, including optional ones gated behind a PEP 508 version requirement, and attach_stub lets type checkers see full static import information via adjacent .pyi stub files while runtime imports stay lazy. The library is intentionally tiny (roughly 400 lines, a single runtime dependency on packaging) and ships with an extensive test suite covering shadowing edge cases, eager-import overrides, and thread-safety under concurrent imports.

What You Get

• attach() for package-internal lazy imports - Replaces manual import submodule statements in __init__.py with a generated __getattr__/__dir__/__all__ trio that imports each submodule or attribute only on first access.
• load() for external dependencies - Returns a proxy module for an external library (e.g. numpy, scipy) that performs the real import only when an attribute is first accessed, with a require= PEP 508 version guard for optional dependencies.
• attach_stub() for type-checker-friendly laziness - Parses an adjacent .pyi stub file via Python’s ast module to derive submodules/submod_attrs automatically, so static type checkers see full imports while runtime stays lazy.
• EAGER_IMPORT escape hatch - An environment variable that disables all lazy behavior at once, useful for debugging import errors or running under tools that don’t tolerate deferred imports.
• Shadow-safe attribute resolution - A custom _ShadowGuardModule module subclass prevents a submodule import from silently overwriting a same-named function attribute (e.g. a max_tree function defined in max_tree.py).

Common Use Cases

• Speeding up import for large scientific packages - Projects like scikit-image lazily attach dozens of subpackages so importing the top-level package stays fast even as the library grows.
• Making optional dependencies truly optional - Libraries load an optional integration only when a user actually calls into it, avoiding a hard ImportError at import time.
• Enforcing minimum versions for lazy dependencies - lazy.load("numpy", require="numpy >=1.24") raises only when the feature needing that version is actually accessed.
• Keeping static typing intact for lazily-loaded packages - attach_stub lets a package stay lazy at runtime while IDEs and mypy still see accurate imports via .pyi files.

Under The Hood

Architecture lazy-loader is a single-module library with three public entry points — attach, load, and attach_stub — built almost entirely from closures and sys.modules/importlib manipulation rather than classes. attach() builds a __getattr__/__dir__/__all__ triplet from caller-supplied submodule and attribute maps, and patches the calling package’s class to _ShadowGuardModule only when it detects a name collision between an attribute and its defining submodule. load() wraps importlib.util.LazyLoader behind a thread lock to hand back a proxy module for an external dependency, optionally checked against a PEP 508 version requirement. attach_stub() is a thin adapter that parses a .pyi file with an ast.NodeVisitor and forwards the result into attach(). There is effectively no internal layering to speak of — the surface is deliberately minimal, and downstream impact from changing the attach() closure would ripple broadly since it underlies the officially adopted Scientific Python SPEC 1 lazy-loading convention used by scikit-image and peer projects.

Tech Stack The project is pure Python (supporting 3.9 through 3.14) with a single runtime dependency, packaging, used only for PEP 508 requirement parsing in _check_requirement. It builds with setuptools>=61.2 via a dynamic version pulled from lazy_loader.__version__, and development tooling runs through spin, the Scientific Python project’s build/test command runner. Dependency groups are split into test (pytest, pytest-cov, coverage[toml]), lint (pre-commit), and dev (changelist, spin) rather than a monolithic dev requirements file.

Code Quality The test suite (tests/test_lazy_loader.py, ~280 lines) exercises eager-vs-lazy behavior, subpackage-import warnings, the shadow-guard collision case, .pyi stub parsing errors, and thread safety under concurrent imports via a separate subprocess-driven test (import_np_parallel.py), backed by realistic fixture packages (tests/fake_pkg, tests/fake_pkg_submodule). Linting runs an unusually broad Ruff rule set (bugbear, isort, pylint subset, simplify, refurb, flake8-pyi) enforced through pre-commit, and CI runs both a test matrix and a dedicated coverage job uploading to Codecov. Error handling is explicit and typed to intent (ModuleNotFoundError, AttributeError, ValueError raised deliberately, never swallowed), though the module carries no static type annotations of its own — a reasonable tradeoff given how much of the code operates through __getattr__ and dynamic module patching.

What Makes It Unique The _ShadowGuardModule trick — intercepting setattr calls the import machinery makes on a package to stop a submodule import from clobbering a same-named function attribute — is a genuinely non-obvious fix for a real quirk in Python’s import system, not something typical lazy-import recipes handle. attach_stub’s reuse of .pyi type-stub files as the source of truth for lazy-import metadata is similarly distinctive: it lets a package satisfy static type checkers and stay lazy at runtime from the same declaration, rather than forcing a choice between the two. The library is also the reference implementation backing the Scientific Python ecosystem’s SPEC 1 lazy-loading standard, giving it outsized real-world reach relative to its size.