backports.zstd

backports.zstd brings PEP-784’s new compression.zstd standard library module to Python 3.10 through 3.13, letting projects adopt Zstandard compression years before they can require Python 3.14. The package tracks CPython’s actual C source for the zstd module rather than reimplementing bindings from scratch, and ships zstd-aware variants of tarfile and zipfile so archives compressed with Zstandard can be read and written the same way they will be once the feature lands in the standard library.

Under the hood it wraps Facebook’s canonical zstd C library (vendored as a git submodule) through a CPython-compatible C extension, with a CFFI-based fallback so PyPy users get the same API. A register_shutil() helper hooks Zstandard support into shutil’s archive format registry, and the whole surface—compress/decompress, ZstdCompressor/ZstdDecompressor, ZstdDict, and the streaming ZstdFile—mirrors what will ship in 3.14 so migrating off the backport later is a one-line import change.

What You Get

• compression.zstd-compatible API - compress, decompress, ZstdCompressor, ZstdDecompressor, and ZstdDict match the upcoming standard-library signatures exactly.
• Zstandard-aware tarfile - a backported tarfile module that reads and writes .tar.zst archives with the same modes (r:zstd) and arguments the stdlib will support.
• Zstandard-aware zipfile - a backported zipfile module with the ZIP_ZSTANDARD compression type for .zip archives.
• shutil integration - register_shutil() registers a zstdtar archive format (and zip Zstandard support) with shutil.unpack_archive/make_archive.
• PyPy support via CFFI - a CFFI-based build path ships the same API for PyPy, not just CPython.

Common Use Cases

• Adopting PEP-784 early - libraries wanting to support Zstandard compression before dropping Python <3.14 support import from backports.zstd conditionally.
• Distributing .tar.zst archives - build pipelines that ship compressed release archives use the backported tarfile module for smaller, faster archives than gzip.
• Reading Zstandard-compressed zip files - tools that consume .zip archives compressed with ZIP_ZSTANDARD need this backport to open them on Python <3.14.
• Cross-version compatibility shims - packages targeting multiple Python versions use the conditional-import pattern shown in the README to support 3.10 through 3.14+ with one code path.

Under The Hood

Architecture The package is a facade over a native extension: src/python/backports/zstd/init.py re-exports compress/decompress/ZstdCompressor/ZstdDecompressor/ZstdDict from the C extension _zstd, while _zstdfile.py layers a streaming ZstdFile/open on top, and tarfile.py/zipfile add zstd-aware archive formats on top of those primitives. _compat.py isolates small polyfills (like os_path_splitroot) needed to run on older Python versions, and _shutil.py registers the new archive formats into stdlib shutil’s format registry. Every layer is deliberately structured to mirror CPython’s own module boundaries so the design goal—byte-for-byte parity with the eventual 3.14 standard library—stays enforceable rather than aspirational.

Tech Stack Runtime code targets Python 3.10 through 3.13 (Python 3.14+ is intentionally blocked in favor of the real stdlib module) with a C extension built via setuptools (setup.py + pyproject.toml, setuptools>=80 backend) that vendors Facebook’s canonical zstd library as a git submodule (src/c/zstd) alongside pythoncapi-compat for cross-version C-API compatibility; an opt-in —system-zstd build flag links the system libzstd instead. PyPy support is provided by a separate CFFI extension built from src/c/cffi. GitHub Actions CI builds wheels across cp310-cp313 and pp310-pp311 targets and sdists, with a dedicated integration job run directly on Python 3.14.

Code Quality The test suite is largely CPython’s own upstream tests for compression.zstd, tarfile, and zipfile (tests/test/test_zstd.py, test_tarfile.py, test_zipfile/test_core.py), copied in with minimal patches for older Python compatibility, plus project-specific tests/test_extension.py, tests/test_shutil.py, and an integration script run against real Python 3.14 in CI to confirm parity. Errors are explicit and typed (ZstdError mirrors the stdlib exception), and CI runs the full suite via python -m unittest discover across every build target—reusing CPython’s own hardened tests gives this backport unusually strong quality assurance for its size.

API Design The public API is intentionally a carbon copy of the not-yet-released stdlib module: the only thing consumers write is a version-gated conditional import (sys.version_info >= (3, 14)), after which the code is identical to what will eventually ship in core Python. That constraint doubles as the project’s documentation strategy—usage examples point directly at the official CPython 3.14 docs for zstd, tarfile, and zipfile rather than duplicating them, keeping the developer experience tied to upstream rather than a bespoke API surface.