cramjam

cramjam is a thin binding layer exposing Rust’s compression ecosystem to Python (via PyO3/maturin) and to JavaScript/WASM (via an npm package), giving callers a single consistent API across Snappy, LZ4, Brotli, Bzip2, Zstd, Gzip, Deflate, zlib, XZ/LZMA, and Blosc2 codecs. Because the codecs are implemented in Rust rather than shelling out to C libraries or pure-Python fallbacks, it installs with no system dependencies and no separate Python or system libraries required.

Each codec module exposes matching compress/decompress functions plus streaming/chunked variants and Rust-backed IO buffer types, so switching between formats is largely a matter of importing a different submodule (cramjam.snappy, cramjam.zstd, etc.) rather than learning a new library’s API. A companion cramjam-cli tool and JS/WASM build extend the same codecs outside pure Python usage.

What You Get

• Consistent compress/decompress functions across nine-plus codecs: Snappy, LZ4, Brotli, Bzip2, Zstd, Gzip, Deflate, zlib, XZ/LZMA, and Blosc2
• Zero system dependencies — pure Rust codec implementations bundled into the compiled wheel, no libzstd/liblz4 etc. required on the host
• Rust-backed IO buffer types (src/io.rs) for streaming compression/decompression without loading entire payloads into memory
• A companion JavaScript/WASM build (cramjam-js) published to npm for using the same codecs in Node.js or the browser
• Type stubs (.pyi files) and a py.typed marker for full static-typing support in Python
• A standalone cramjam-cli tool for command-line compression/decompression outside of Python

Common Use Cases

• Reading/writing compressed columnar data formats (e.g. Parquet with Snappy/Zstd/LZ4 codecs) without needing system compression libraries installed
• High-throughput data pipelines needing fast compression with a single dependency covering multiple codecs instead of separate bindings per format
• Cross-format compression tooling where the codec must be chosen at runtime (e.g. supporting whichever format an upstream file uses)
• Browser or Node.js applications needing the same compression codecs as a Python backend, via the cramjam-js WASM build

Under The Hood

Architecture - The Rust workspace defines one source module per codec (src/snappy.rs, src/lz4.rs, src/brotli.rs, src/bzip2.rs, src/zstd.rs, src/gzip.rs, src/deflate.rs, src/zlib.rs, src/xz.rs, src/blosc2.rs, plus src/igzip.rs/ideflate.rs/izlib.rs for Intel ISA-L accelerated variants), each wrapping an underlying Rust compression crate behind a shared compress/decompress function signature; src/io.rs provides Rust-native buffer/stream types shared across codecs, and the Python package (src/cramjam/) is generated via PyO3/maturin bindings with matching .pyi stubs per submodule. A sibling cramjam-js workspace member targets WASM for the npm package, reusing the same core Rust codec code. Tech Stack - Rust workspace built with maturin (declared via [[metadata.maturin]] in Cargo.toml) producing a cramjam-python compiled extension; the JS/WASM side uses wasm-bindgen-style tooling via cramjam-js, and the repo also ships package.json/uv.lock reflecting its multi-language build (Cargo, pip/maturin, npm). Code Quality - Python-side tests cover buffer views, per-codec variants, cross-codec integration against real compressed fixture files (tests/data/integration/plaintext.txt.{bz2,lz4,gz,zst,br,snappy,lzma}), and Rust IO behavior (test_buffer_view.py, test_variants.py, test_integration.py, test_rust_io.py), plus a test_cramjam_pyodide.js file exercising the WASM build; CI badges in the README indicate this is checked on every push. API Design - Every codec submodule exposes the same compress(data)/decompress(data) shape, so switching from one format to another (e.g. cramjam.snappy to cramjam.zstd) requires no new API to learn, and bundled .pyi stubs give IDE autocompletion and static-type checking out of the box.