pbjson

Pbjson is a small family of Rust crates — pbjson, pbjson-build, and pbjson-types — that automatically generate serde Serialize and Deserialize implementations for structs produced by prost, the Protocol Buffers code generator. Rather than hand-writing JSON (de)serialization logic for every message, pbjson-build reads the FileDescriptorSet emitted by prost-build during a crate’s build.rs step and generates JSON mapping code that follows the official Protocol Buffers JSON mapping specification — camelCase field names, base64-encoded bytes, string-encoded 64-bit integers, and enum name/number interchangeability all included.

The runtime crate, pbjson, supplies the small set of shared helpers (base64 handling, numeric string coercion) that the generated code depends on, while pbjson-types provides pbjson-compatible implementations of the well-known protobuf types (Duration, Timestamp, Struct, Any, and others) so consumers don’t need to write their own. It’s maintained by InfluxData and used internally in projects like InfluxDB IOx wherever prost-generated types need to round-trip through JSON.

What You Get

• A pbjson-build Builder API that hooks into build.rs after prost-build to emit a <package>.serde.rs file containing generated Serialize/Deserialize impls
• Protobuf JSON mapping compliance out of the box — camelCase field renaming, base64-encoded bytes, string-encoded int64/uint64, and enum name-or-number deserialization
• A pbjson-types crate providing ready-made JSON-serializable versions of the well-known types (Duration, Timestamp, Struct, Value, Any, etc.)
• Shared pbjson runtime helpers (NumberDeserialize, BytesDeserialize) used by the generated code for lenient JSON number and bytes parsing
• Builder options for common customizations — ignore_unknown_fields, emit_fields, use_integers_for_enums, preserve_proto_field_names, btree_map, and extern_path

Common Use Cases

• Exposing a JSON REST or gRPC-gateway API alongside a primarily-protobuf gRPC service without writing separate JSON DTOs
• Building configuration or storage formats around protobuf schemas that also need a human-readable JSON representation
• Round-tripping protobuf messages through JSON in integration tests, debugging tools, or logging
• Interoperating with JSON-only clients (browsers, curl, other languages) that consume the same protobuf-defined data model

Under The Hood

Architecture Pbjson-build implements a straightforward compiler-style pipeline that runs inside a consumer’s build.rs. descriptor.rs parses the FileDescriptorSet bytes emitted by prost-build into a DescriptorSet/Package model; resolver.rs resolves fully-qualified protobuf type paths (including registered extern_path overrides) to their generated Rust types; message.rs walks each message’s fields and normalizes them into an intermediate representation carrying JSON-mapping-specific metadata (camelCase names, default values, oneof handling); and generator.rs turns that representation into TokenStreams implementing serde::Serialize/Deserialize. The public Builder struct in lib.rs orchestrates the pipeline — register_descriptors feeds it input, chained configuration methods (exclude, btree_map, ignore_unknown_fields, etc.) set generation options, and build() writes the generated code to OUT_DIR. This is a classic build-time codegen architecture: correctness of the resolver and message-normalization stages is what everything downstream depends on, and a change to the intermediate field representation would ripple through the generator.

Tech Stack The workspace targets Rust 2024 edition and is built around the prost/prost-build/prost-types 0.14 family for protobuf parsing and descriptor generation, with syn/quote/proc-macro2 (via prost-build’s TokenStream output) used for code generation. The runtime crate depends only on serde 1.0 and base64 0.22; pbjson-types additionally depends on chrono for Duration/Timestamp conversions. The build-only crate uses heck for case conversion (snake_case to camelCase) and itertools for iteration helpers. There is no async runtime, database, or web framework involved — this is a pure codegen/serialization library consumed from other crates’ build.rs and Cargo.toml.

Code Quality CI (CircleCI) runs cargo fmt --check, cargo clippy --all-targets --workspace -- -D warnings, cargo audit, cargo doc, and cargo test --workspace repeated across seven different feature-flag combinations (ignore-unknown-fields, btree, emit-fields, use-integers-for-enums, preserve-proto-field-names, ignore-unknown-enum-variants), plus a vendor job that regenerates pbjson-types’ descriptor file from source .proto files and fails the build if it drifts from the committed version — an unusually thorough consistency check for a codegen crate. Tests live both as inline #[test] unit tests in descriptor.rs/resolver.rs/pbjson/lib.rs and as integration tests in the dedicated pbjson-test crate, which compiles real .proto fixtures (including edge cases like duplicate_name.proto and escape.proto) and asserts against the generated serde output. Error handling favors std::io::Result and explicit Error::other(...) construction over panics in the build-time path.

API Design The public surface is a small, chainable Builder with self-documenting method names (ignore_unknown_fields(), preserve_proto_field_names(), btree_map(paths)) that map directly onto Protobuf JSON mapping spec options, keeping the day-to-day API easy to scan. Getting started, however, requires wiring three crates together (pbjson, pbjson-build, pbjson-types) plus prost/prost-build, and copying a non-trivial build.rs snippet — the crate-level rustdoc in pbjson-build/src/lib.rs supplies this snippet directly rather than relying on a separate docs site, which lowers the onboarding cost but still asks for real protobuf/build-script familiarity. The top-level README is intentionally minimal and defers entirely to the pbjson-build docs.rs page for usage instructions.