prost

Prost is a Protocol Buffers implementation for the Rust language, generating clean, idiomatic Rust code directly from proto2 and proto3 .proto files. Rather than emitting large generated modules full of boilerplate, Prost leans on Rust’s derive-macro system so .proto messages become plain structs annotated with #[derive(Message)], oneofs become enums, and comments from the source .proto files are preserved in the generated docs.

Maintained under the Tokio project umbrella, Prost is the de facto Protobuf implementation for the Rust ecosystem and is the code-generation layer beneath gRPC libraries like tonic. It supports no_std environments, preserves unknown enum values on deserialization (important for forward-compatible wire formats), and uses the bytes crate’s Buf/BufMut abstractions instead of std::io::{Read, Write} for zero-copy-friendly encoding and decoding.

What You Get

• The prost runtime crate: Message, Enumeration, and Oneof derive macros plus the Message trait (encode, decode, encoded_len, and buffer-based variants) for any type annotated with #[derive(Message)]
• prost-build, a build-script library that invokes protoc (or a pure-Rust protoc-free path) to compile .proto files into Rust modules during cargo build
• prost-types, ready-made Rust bindings for the Protobuf well-known types (Any, Timestamp, Duration, Struct, etc.)
• Preservation of .proto comments as Rust doc comments on generated types, plus package-to-module mapping (package foo.bar becomes the foo::bar Rust module)
• no_std support (disable the std feature) so Prost can be used in embedded and constrained environments
• A BTreeMap configuration option in prost-build for deterministic, allocation-conscious map encoding instead of HashMap

Common Use Cases

• Generating Rust request/response types for gRPC services when paired with tonic as the RPC transport
• Encoding/decoding a stable, versioned wire format for cross-service or cross-language communication (Rust services talking to Go, Java, Python, or C++ peers over Protobuf)
• Persisting structured records in a compact binary format for storage or message-queue payloads where JSON’s size and stringliness are undesirable
• Building no_std firmware or embedded systems that need a compact, cross-language-compatible message format without pulling in std
• Emitting a FileDescriptorSet for runtime introspection tooling that needs the original .proto schema shape alongside generated code

Under The Hood

Architecture: The workspace splits cleanly along a build-time/runtime boundary. prost-build (a separate crate invoked from a consuming project’s build.rs) shells out to protoc to parse .proto files into a FileDescriptorSet, then walks that descriptor tree to emit Rust source implementing the Message trait for each message type. At runtime, the prost crate itself is deliberately small: src/message.rs defines the core Message trait (encode_raw, merge_field, encoded_len, plus default-provided encode/decode/encode_to_vec methods), src/encoding.rs (1,462 lines) implements the actual varint/length-delimited wire encoding for every Protobuf scalar and collection shape via macro-generated encode/merge/encoded_len function families, and src/types.rs plus the separate prost-types crate supply the well-known-type bindings. prost-derive is the proc-macro crate that reads struct/enum field attributes (emitted by prost-build) and generates the actual Message/Enumeration/Oneof trait implementations, closing the loop between build-time codegen and runtime behavior. Tech Stack: Pure Rust workspace (Cargo.toml lists prost, prost-build, prost-derive, prost-types, plus dedicated tests, tests-2015/2018/2024, and tests-no-std crates for edition and no_std compatibility matrices). The only runtime dependency of the core prost crate is bytes (for Buf/BufMut); prost-derive is an optional dependency gated behind the derive feature (on by default). MSRV is pinned at Rust 1.85 with a documented six-month rolling policy. Dev-dependencies include criterion for benchmarking varint encoding and proptest/rand for property-based testing. Code Quality: The crate carries a dedicated tests workspace member (with prost-path, reexported-prost, and single-include sub-crates) exercising re-export and path-independence scenarios, alongside edition-specific test crates (tests-2015, tests-2018, tests-2024) and a tests-no-std crate proving the no_std feature actually compiles without std. A fuzz/ directory plus FUZZING.md and KANI.md document both libFuzzer-based fuzz targets and formal verification via the Kani model checker — an unusually rigorous quality bar for a serialization library where malformed input handling is a real attack surface. DecodeError in src/error.rs tracks a ‘best effort’ error description plus a (message, field) name stack for nested decode failures, giving actionable error context rather than an opaque failure. CI (.github/workflows/ci.yml, cifuzz.yml) runs the full matrix on every push. API Design: The public surface centers on one trait (Message) and three derive macros, keeping the mental model small: annotate a struct, call .encode()/Message::decode(). Scalar-to-Rust-type mapping (int32 to i32, bytes to Vec<u8>, etc.) follows Rust conventions rather than inventing new wrapper types, and generated oneofs become plain Rust enums matched with match. The tradeoff for this simplicity is that some internal-use methods (encode_raw, merge_field) are #[doc(hidden)] but technically public, and the README explicitly flags a deprecated DecodeError::new that leaked into public API by accident — an honest, documented rough edge rather than a silent trap.