BLAKE3

BLAKE3 is the official Rust crate implementing the BLAKE3 cryptographic hash function — a single, variant-free algorithm that is significantly faster than MD5, SHA-1, SHA-2, SHA-3, and BLAKE2 while remaining cryptographically secure against length-extension attacks. Its internal Merkle tree structure makes it highly parallelizable across threads and SIMD lanes and enables verified streaming and incremental updates.

The crate ships hand-optimized SIMD implementations for SSE2, SSE4.1, AVX2, AVX-512, NEON, and WASM with automatic runtime CPU feature detection on x86, plus an optional rayon-powered multithreading mode. Beyond plain hashing, BLAKE3 doubles as a PRF, MAC, KDF, and XOF, giving it a broad range of cryptographic use cases from checksumming to key derivation.

What You Get

• A Hasher struct implementing incremental, streaming hashing with update/finalize and impl Write/Read/Seek support behind the std feature
• Automatic runtime CPU feature detection on x86 selecting between SSE2, SSE4.1, AVX2, and AVX-512 implementations
• SIMD-optimized NEON and WASM SIMD code paths for ARM and WebAssembly targets
• Keyed hashing and key derivation function (KDF) modes in addition to plain and extendable-output (XOF) hashing
• Optional rayon-based multithreaded hashing via update_rayon/update_mmap_rayon for large inputs

Common Use Cases

• Content-addressed storage and deduplication systems that need a fast, collision-resistant hash for large files
• Checksumming and integrity verification tools where hashing throughput on multi-core hardware matters, as with the companion b3sum CLI
• Key derivation for applications needing a fast KDF, using BLAKE3’s dedicated derive_key mode instead of a separate KDF library
• Merkle-tree-based verified streaming, where a file can be hashed and verified incrementally without holding the full content in memory

Under The Hood

Architecture The blake3 crate’s core lives in src/lib.rs, src/platform.rs, and src/guts.rs, structured around a portable implementation (src/portable.rs) plus a set of hand-written SIMD backends (src/rust_sse2.rs, src/rust_sse41.rs, src/rust_avx2.rs, and FFI bridges to C/assembly for AVX-512 and NEON in src/ffi_avx512.rs/src/ffi_neon.rs) selected at runtime via the cpufeatures-based detection in platform.rs. The algorithm itself processes input as a Merkle tree of 1KiB chunks, which is what enables both SIMD/thread parallelism across chunks and incremental/streaming updates without buffering the whole input; src/join.rs implements the (optional, rayon-backed) parallel tree-reduction logic.

Tech Stack A Rust 2024-edition crate with a build script (build.rs) that compiles vendored C/assembly SIMD kernels for platforms where Rust intrinsics aren’t used, plus a small dependency surface (arrayref, arrayvec, constant_time_eq, cpufeatures, with optional rayon, serde, mmap, and zeroize features). The repository is a monorepo also containing the b3sum CLI crate, a C implementation under c/, and a separate minimal reference implementation used for spec verification.

Code Quality The crate includes an extensive src/test.rs plus a test_vectors/ directory of cross-language test vectors shared with the C implementation and other language ports, reflecting rigor appropriate for cryptographic code; CI (implied by the multi-platform SIMD backend structure) must exercise each SIMD path plus the portable fallback for correctness parity. As a security-sensitive primitive, the crate is conservative about unsafe usage, isolating it to well-reviewed SIMD/FFI boundary code rather than scattering it through the public API.

API Design The public API is intentionally small and safe-by-default: blake3::hash(&[u8]) for one-shot hashing, blake3::Hasher::new() for streaming, and dedicated constructors (new_keyed, new_derive_key) for the keyed/KDF modes — a design that keeps common cases to one line while still surfacing the more specialized cryptographic modes explicitly rather than through hidden flags. Feature flags (std, rayon, mmap, serde, neon) let consumers opt into only the capabilities (multithreading, serialization, no_std) they need.