data-encoding

data-encoding provides efficient, customizable base-conversion encoding and decoding for Rust, covering standard encodings (hex, base32, base64 and their variants) as well as fully custom bit-width encodings defined at runtime or compile time. It supports padding, canonical validation, in-place and partial decoding, character translation, bit-order selection, and ignoring characters like newlines while decoding.

The repository is a small workspace: the data-encoding library itself (the flagship, importable crate), a data-encoding-macro crate for defining custom encodings at compile time as const values, and a data-encoding-bin command-line tool for defining and using encodings from the shell. It works in no_std and even no_alloc environments, making it usable in embedded and other constrained contexts.

What You Get

• Predefined Encoding constants for hex, base32 (standard and hex-alphabet variants), and base64 (standard, URL-safe, no-padding) forms
• A Specification builder for defining fully custom encodings with your own symbol set, padding, casing, and bit-order rules
• In-place encode/decode functions that avoid extra allocation, plus partial decode functions for error recovery
• no_std and no_alloc support for embedded or otherwise constrained environments
• A companion data-encoding-macro crate for defining custom encodings as compile-time const values, and a data-encoding-bin CLI for ad-hoc encode/decode from the shell

Common Use Cases

• Encoding binary data (hashes, tokens, keys) into URL-safe or filename-safe base64/base32 text
• Decoding untrusted input encoded as hex/base32/base64 with strict canonical validation to reject malformed data
• Defining a project-specific custom alphabet encoding (e.g. Crockford base32) via the Specification API or the compile-time macro
• Performing encode/decode operations in no_std embedded firmware where the standard library isn’t available

Under The Hood

Architecture - The entire encode/decode implementation lives in a single, carefully structured lib.rs (~2,800 lines) organized around the Encoding type (an opaque, validated table of translation rules) and a Specification builder that validates and compiles user-supplied symbol/padding/bit-order rules into an Encoding. The workspace additionally includes data-encoding-macro (compile-time encoding definitions via proc macros) and data-encoding-bin (a thin CLI wrapper), keeping the core allocation-free algorithm decoupled from both compile-time and command-line convenience layers.

Tech Stack - The core data-encoding crate has zero required dependencies and supports three tiers of environment: full std, alloc-only (no_std with heap), and fully allocation-free, selected via Cargo features. It targets Rust edition 2018 with a conservative MSRV of 1.48, prioritizing broad compatibility over using newer language features.

Code Quality - Correctness is validated through a dedicated lib/tests integration-test crate plus an active fuzzing setup (lib/fuzz) referenced in the README’s fuzzing-status badge, which matters for an encoding library where malformed/adversarial input handling is a core correctness concern. The crate’s [lints] section enables clippy::undocumented-unsafe-blocks, missing-debug-implementations, and missing-docs warnings, signaling a deliberately strict internal quality bar despite the project’s otherwise low commit cadence.

API Design - The predefined constants (BASE64, BASE32, HEXLOWER, etc.) cover the common case with a single .encode()/.decode() call, while the Specification builder makes custom encodings (e.g. Crockford base32) a matter of setting a few struct fields rather than reimplementing a codec from scratch — a deliberate design choice that trades a slightly larger initial API surface for avoiding the need to hand-roll bit-packing logic for non-standard alphabets.