jsonc-parser

jsonc-parser is a Rust crate that parses JSONC — JSON extended with line and block comments, trailing commas, single-quoted strings, hexadecimal and unary-plus numbers, and unquoted or loosely-quoted object keys. It exposes three output modes so callers only pay for what they need: parse_to_value for a simple JsonValue tree, parse_to_ast for a positioned AST that separately tracks comments and tokens, and (behind the cst cargo feature) a mutable concrete syntax tree with a CstRootNode object/array API for getting, setting, and appending values while preserving the original formatting and comments on serialization.

The crate has no required dependencies; optional cargo features add serde/serde_json interop for deserializing JSONC directly into typed structs, preserve_order (via indexmap) for stable object key ordering, fast_hash (via rustc-hash) for faster parsing of trusted input, and error_unicode_width for more accurate error column numbers in wide-character text. It’s maintained by dprint, the Rust-based code formatter, which uses it to read and rewrite JSONC configuration files without disturbing developer comments or formatting.

What You Get

• A parse_to_value function returning a simple JsonValue for straightforward reads
• A parse_to_ast function returning a positioned AST with comments and tokens collected separately
• An optional cst feature exposing a CstRootNode with get/set_value/append methods for programmatic, format-preserving edits
• Optional serde feature integration via parse_to_serde_value to deserialize JSONC straight into typed structs
• ParseOptions to toggle comments, trailing commas, missing commas, single-quoted strings, hex numbers, unary-plus numbers, and loose property names individually
• Typed, exhaustive parse errors (errors.rs) instead of panics, with an opt-in unicode-width feature for accurate column numbers

Common Use Cases

• Reading and validating JSONC configuration files (e.g. tsconfig.json-style files) that contain developer comments
• Programmatically rewriting a config file’s values while preserving the rest of the file’s comments and formatting
• Deserializing loosely-formatted JSON input directly into application structs via the serde integration
• Building developer tools (formatters, linters, codemods) that need to inspect or transform JSONC without a full round-trip through a generic AST library

Under The Hood

Architecture The crate is layered as scanner -> parser -> output. scanner.rs tokenizes raw text into Tokens while handling comments, quoting styles, and numeric literal variants; parser.rs wraps the scanner in a shared JsoncParser that centralizes comment-skipping, nesting-depth tracking, and comma/separator rules reused by two independent output stages, parse_to_value.rs (a plain JsonValue tree) and parse_to_ast.rs (a positioned AST that keeps comments and tokens as first-class data). A third, much larger mode lives in cst/mod.rs (~4,400 lines, gated behind the cst feature): it builds a mutable, Rc/RefCell-linked concrete syntax tree over the AST with a CstRootNode object/array API (get, set_value, append) supporting in-place, format-preserving edits — this module duplicates positional bookkeeping (Ranged, input.rs) on top of what the AST already tracks, making it the piece most exposed to churn if the token/AST model changes.

Tech Stack Built on the 2024 Rust edition with zero required dependencies — everything beyond the core parser is opt-in via cargo features: indexmap for preserve_order, rustc-hash for fast_hash, serde/serde_json for typed deserialization, and unicode-width for accurate error columns. It’s a pure in-memory text-to-value/AST/CST library with no async runtime or I/O, formatted with dprint plus rustfmt, and benchmarked with a dedicated benches/bench.rs suite.

Code Quality Correctness is driven by a large, data-driven fixture suite under tests/specs/{array,comments,encoding,object,string} — paired input/expected-output files compared with pretty_assertions::assert_eq and exercised against both the AST and (feature-gated) CST parsers, giving broad coverage of edge-case syntax rather than hand-written unit tests. Errors are represented as a typed, exhaustive enum (errors.rs) instead of panics, and the crate root enforces #![deny(clippy::print_stderr)] / #![deny(clippy::print_stdout)]. CI runs cargo test across multiple feature combinations (serde, preserve_order, all-features) in both debug and release.

API Design The standout design choice is letting callers pick their representation cost: a plain value tree for simple reads, a positioned AST when comments and token spans matter, or a full mutable CST when the goal is editing a JSONC file and re-emitting it with formatting and comments intact — a capability generic JSON crates like serde_json don’t offer. ParseOptions exposes each syntax relaxation (comments, trailing commas, single quotes, hex numbers, etc.) as an individual boolean, so consumers can dial the parser from fully strict JSON to fully permissive JSONC without writing their own pre-processor.