bumparaw-collections

bumparaw-collections is a small Rust crate from the Meilisearch team that combines bump allocation (bumpalo) with serde_json’s RawValue to manipulate JSON while parsing it only minimally and gradually. Instead of eagerly deserializing an entire document, its collections keep references into the underlying bytes and materialize values lazily.

The crate ships arena-backed maps and vectors, a partially-parsed Value type, a string interner, and a bitpacked Bbbul structure, all designed for low-level, allocation-conscious work with large JSON payloads inside a bumpalo::Bump arena.

What You Get

• RawMap and FrozenMap — bumpalo-backed JSON object maps with iteration and deserialization support.
• RawVec — an arena-backed vector for JSON arrays.
• Value — a partially-parsed JSON value enum that references the underlying data instead of copying it.
• Bbbul / FrozenBbbul — bitpacked structures for compact, low-level storage.
• A simple str interner and allocator_api2::Allocator implementations for bumpalo objects.

Common Use Cases

• Parsing and manipulating large JSON documents with minimal allocation inside a bump arena.
• Deferring full JSON deserialization by holding onto RawValue references until a field is needed.
• Interning repeated strings to cut memory usage in JSON-heavy pipelines.
• Building search-index or database internals that process JSON at high throughput.

Under The Hood

Architecture — The crate is organized around arena-backed containers: map.rs/map/ implement RawMap and FrozenMap over hashbrown with a bumpalo allocator, vec.rs/vec/ implement RawVec, and value.rs defines a partially-parsed Value enum whose variants (String, Array, Object) borrow from the 'bump arena rather than owning copies. bbbul.rs adds a bitpacked structure and alloc.rs provides allocator_api2::Allocator implementations bridging bumpalo to the allocator API. lib.rs re-exports the primary types (Bbbul, RawMap, RawVec, Value).

Tech Stack — Rust 2021 with dependencies on bumpalo (arena allocation), serde/serde_json with the raw_value and preserve_order features, hashbrown, allocator-api2, and bitpacking. It is MIT-licensed and categorized under algorithms and data-structures on crates.io.

Code Quality — The crate enforces #![warn(missing_docs)] and documents every public module, so despite an empty README the rustdoc is the real reference. Test coverage exists in a dedicated test.rs module and inline #[test] blocks in bbbul.rs, with rand as a dev-dependency for property-style checks. Module boundaries are clean and lifetime-parameterized around the 'bump arena.

API Design — The public surface mirrors familiar collection names (RawMap, RawVec) so the arena and lazy-parsing behavior is the only new concept to learn. Working with it requires understanding bumpalo lifetimes, which raises the entry bar somewhat, but the payoff is precise control over allocation. It is a low-level building block rather than a high-level convenience API.