rust-snappy

snap is a from-scratch Rust port of Google’s Snappy compression format, ported from both the reference C++ implementation and the Go implementation. It provides both the raw Snappy block format (raw::Encoder/raw::Decoder) for single-shot compression, and the Snappy frame format (read::FrameDecoder/write::FrameEncoder) for streaming compression and decompression over any std::io::Read/Write type.

Unlike crates that bind to the C++ Snappy library via FFI, snap has zero runtime dependencies and compiles as pure, portable Rust. Its authors validate output byte-for-byte against the reference C++ implementation and benchmark it against both the C++ and Go implementations, consistently landing within a few percent of native performance on x86_64.

What You Get

• Streaming frame API — read::FrameDecoder and write::FrameEncoder implement std::io::Read/Write, so any stream can be wrapped for on-the-fly Snappy compression or decompression.
• Raw block API — raw::Encoder/raw::Decoder give lower-level, single-shot access to the Snappy block format for callers who need to manage buffers themselves.
• Zero runtime dependencies — the core crate compiles as pure Rust with no FFI or C toolchain requirement, unlike bindings-based Snappy crates.
• Typed, structured errors — a dedicated Error enum (TooBig, BufferTooSmall, HeaderMismatch, etc.) with a built-in conversion to io::Error for use with ?.
• The szip companion tool — a gzip-like CLI (cargo install szip) built on top of the library for compressing and decompressing files from the command line.

Common Use Cases

• Compressing log or event streams — wrap a Write sink in write::FrameEncoder to compress data as it’s produced, without buffering the whole payload in memory.
• Reading compressed data files — wrap a Read source in read::FrameDecoder to transparently decompress Snappy-framed files or network streams.
• Interoperating with other Snappy implementations — because output is validated byte-for-byte against the C++ reference, data compressed by this crate can be read by any conformant Snappy consumer (Go, Java, Python bindings, etc.) and vice versa.
• Ad-hoc file compression via szip — using the bundled CLI for gzip-style compress/decompress workflows without writing any Rust code.

Under The Hood

Architecture The crate is layered cleanly: bytes.rs provides low-level, unsafe raw memory-copy primitives used for performance; compress.rs and decompress.rs implement the raw Snappy block format on top of those primitives, exposed publicly through raw::Encoder/raw::Decoder; frame.rs builds the streaming Snappy frame format (chunked, CRC32-checksummed) on top of the raw block layer; and read.rs/write.rs wrap the frame layer in std::io::Read/Write adapters (FrameDecoder/FrameEncoder) that most consumers use directly. Dependencies flow in a single direction — IO adapters depend on framing, framing depends on the raw codec, the raw codec depends on the byte-copy primitives — with no circular coupling, so a change to the core Encoder/Decoder abstraction would cascade predictably outward through frame.rs into the public IO wrappers.

Tech Stack The main snap crate has zero runtime dependencies (edition 2018), keeping it pure Rust with no C toolchain requirement. It lives in a Cargo workspace alongside a bench crate (Criterion-based benchmarks comparing against the C++ and Go implementations), a test crate (integration tests that can optionally link the reference C++ Snappy library via a cpp feature), and szip, an installable CLI built on the library. CI runs on GitHub Actions using cross to test on 32-bit and big-endian targets in addition to the usual platforms, giving reasonable confidence in portability for a codec crate.

Code Quality There are no inline #[test] functions in the library source itself — all correctness testing lives in the separate test crate, which runs several hundred lines of round-trip and error-path tests against a fixed corpus of real-world text files and known-malformed Snappy payloads, plus doctests executed directly against the README via doc_comment::doctest!. Errors are represented as a structured, documented Error enum rather than strings, with a From<Error> for io::Error conversion for idiomatic ? usage. #![deny(missing_docs)] enforces documentation on every public item. Performance-critical paths use unsafe raw-pointer operations in a handful of files, a deliberate and narrowly scoped tradeoff typical of compression codecs rather than general carelessness.

What Makes It Unique Most Rust Snappy bindings at the time this crate was written wrapped the C++ reference implementation via FFI, adding a build-time C toolchain dependency; snap instead reimplements the algorithm from scratch in safe-by-default Rust, validates its output byte-for-byte against that same C++ reference, and benchmarks within a small margin of native performance on typical workloads. It’s also one of the few pure-Rust implementations to include the full streaming frame format (chunked, checksummed) rather than just the raw block format, which the README notes some contemporary pure-Rust alternatives lacked.