python-tblib

tblib is a small, dependency-free Python library for serializing tracebacks and exceptions so they survive pickling — the core building block for propagating real, re-raisable exceptions across process boundaries in multiprocessing, concurrent.futures, Celery, Dask, and similar systems.

Where Python’s stdlib pickle protocol silently drops the traceback (or fails outright) when an exception crosses a process, tblib patches exception classes via copyreg so pickle.dumps(sys.exc_info()) — or pickling the exception itself with its __cause__/__context__ chain intact — round-trips into an object you can raise in the parent process, complete with a synthetic but walkable traceback. It also offers pickling-free paths: reconstructing a Traceback from a plain dict (to_dict/from_dict, JSON-safe) or from a parsed traceback string (from_string), for when you don’t trust or don’t want pickle at all.

What You Get

• pickling_support.install() - Monkeypatches copyreg so exceptions and their tracebacks pickle correctly, either globally, for specific classes, or for specific instances.
• Traceback wrapper class - A serializable stand-in for the builtin traceback object, with .as_traceback() to convert back into something you can raise.
• to_dict() / from_dict() - JSON-safe traceback serialization that doesn’t touch pickle at all.
• from_string() parser - Reconstructs a walkable traceback from a plain-text stack trace, with a lenient strict=False mode for messy input.
• tblib.decorators.reraise and Error/return_error - Helpers for the common “call a function, capture its exception, reraise it later in a different context” pattern.

Common Use Cases

• Multiprocessing worker pools - Wrap worker functions with return_error/apply_with_return_error so a Pool.map() failure comes back as a real, re-raisable exception with the original traceback instead of a flattened string.
• Task queue and distributed systems - Celery, Dask.distributed, and similar frameworks use tblib to propagate task failures from remote workers back to the caller with an intact traceback.
• Test frameworks reporting remote failures - pytest-xdist and similar tools rely on tblib to preserve tracebacks when tests run in a subprocess or separate worker.
• Debugging tools that store or transmit exceptions - to_dict()/from_dict() let you log or ship a traceback as plain JSON when pickling is undesirable or the boundary isn’t a Python process at all.

Under The Hood

Architecture tblib is a small, three-module package with clean separation of concerns: tblib/__init__.py defines minimal, serializable stand-ins for the builtin Code, Frame, and Traceback objects (a plain traceback isn’t picklable, so these classes carry over just enough state — filename, line number, function name, a trimmed set of globals/locals — to reconstruct one), tblib/pickling_support.py layers copyreg registration on top so real exceptions and TracebackType instances route through these serializable forms transparently, and tblib/decorators.py adds an ergonomic reraise/Error/return_error layer for the common “capture in a worker, reraise in the caller” pattern. There’s no framework and no external coupling — each module builds directly on the one below it, and the reconstruction path (Traceback.as_traceback()) works by compiling a tiny stub function per frame and re-raising through it via exec, which is an unusual but effective way to synthesize a real, walkable traceback object at runtime.

Tech Stack Pure standard-library Python — no runtime dependencies at all, relying only on copyreg, pickle, functools, and types.TracebackType. It targets Python 3.9 through 3.14 on CPython and PyPy. The build uses setuptools with pyproject.toml-only configuration, ruff for linting and formatting (a broad rule set including bugbear, bandit, and pyupgrade), pre-commit hooks, and tox for cross-version testing; test-only dependencies are pytest, pytest-benchmark, and twisted.

Code Quality The test suite (tests/test_tblib.py, test_pickle_exception.py, test_perf.py, plus targeted regression tests like test_issue30.py and test_issue65.py) covers both unit behavior and specific historical bug reports, and the README itself is doctested as part of the docs build, giving the public API executable, always-current examples. CI runs via GitHub Actions across the supported Python matrix with coverage reported to Coveralls and Codecov. Error handling is deliberate rather than defensive — the library explicitly special-cases OSError and ExceptionGroup reduce behavior and keeps a documented compatibility shim for pickle archives produced by older tblib versions.

API Design The public surface is intentionally tiny: one install() call to opt in, called globally, as a class decorator, or against specific instances, plus a Traceback class with to_dict()/from_dict()/from_string()/as_traceback() for pickle-free use. That narrowness is the point — tblib solves exactly one problem (tracebacks aren’t picklable) and refuses to grow beyond it, which is likely why it has quietly become a dependency of larger, well-known distributed-systems tools; the tradeoff is that documentation leans on the README’s doctested examples rather than a fuller external doc site.