pyjson5

pyjson5 is a JSON5 serializer and parser for Python 3 written in Cython and compiled to a C++ extension for speed. It implements the full JSON5 1.0.0 specification (comments, trailing commas, unquoted keys, single-quoted strings, and more) while remaining a strict superset that also reads standard RFC 8259 JSON. The encoder produces ASCII-only output with apostrophes, ampersands, and angle brackets escaped, making its results safe to embed directly inside HTML <script> templates without further sanitization.

Rather than one function that branches on input type, pyjson5 exposes narrow, purpose-built entry points: decode/decode_utf8/decode_latin1/decode_buffer/decode_callback/decode_io for parsing, and matching encode_* variants for serialization, plus loads/load/dumps/dump shims for drop-in compatibility with the standard library’s json module. It has zero runtime dependencies, ships typed stubs (py.typed), and publishes prebuilt wheels — including free-threaded (PEP 703) builds for Python 3.13t/3.14t — across Linux, macOS, and Windows.

What You Get

• Full JSON5 1.0.0 spec support (comments, trailing commas, unquoted keys, single-quoted and multi-line strings) plus strict RFC 8259 JSON compatibility
• Six decode entry points covering str, bytes/Latin-1, bytes/UTF-8, buffer-protocol objects, callback-driven streaming, and file-like IOBase input
• ASCII-safe, HTML-injection-resistant encoder output with escaped apostrophes, ampersands, and angle brackets
• Zero runtime dependencies and prebuilt wheels for Linux/macOS/Windows, including free-threaded CPython 3.13t/3.14t builds
• json-module-compatible load/loads/dump/dumps shims for drop-in migration
• Configurable maximum nesting depth (maxdepth) to guard against pathological or adversarial input

Common Use Cases

• Parsing human-edited configuration files that use JSON5’s comments and trailing commas instead of strict JSON
• Safely serializing server-side data for direct interpolation into inline HTML <script> blocks
• High-throughput JSON parsing/serialization where the stdlib json module’s pure-Python implementation is a bottleneck
• Streaming decode of multiple JSON5 documents back-to-back from a single socket or file handle via decode_io/decode_callback with some=True
• Zero-copy decoding directly from memoryview/bytearray buffers via decode_buffer

Under The Hood

Architecture The library compiles to a single Cython extension module (pyjson5.pyx) that aggregates roughly twenty .pyx include files under src/, separating decoder logic (_decoder.pyx, _readers.pyx, _reader_ucs.pyx, _reader_callback.pyx), encoder logic (_encoder.pyx, _writers.pyx, _writer_reallocatable.pyx, _writer_callback.pyx, _writer_noop.pyx, _encoder_options.pyx), a typed exception hierarchy (_exceptions.pyx plus decoder/encoder-specific raise helpers), Unicode category tables pre-generated from the Unicode Character Database, and a _legacy.pyx shim exposing json-compatible functions. The public API is centralized in _exports.pyx. Both decode and encode fan out into several narrowly-scoped entry points that share one core algorithm each — a strategy-pattern style design over a small number of core routines rather than a single type-sniffing function, and rather than duplicated logic per input/output kind.

Tech Stack Built with Cython (constrained to >=3.1,<4) compiling to C++11 via setuptools’ Extension/build_ext, with aggressive optimization flags (-O3, -fomit-frame-pointer). The encoder embeds a vendored Dragonbox implementation for fast float-to-string conversion, and Unicode category tables are generated ahead of time from the Unicode 17.0.0 UCD rather than resolved at runtime. wheels.yml uses cibuildwheel to publish prebuilt wheels across the OS/Python matrix, including free-threaded builds opted into via Cython’s freethreading_compatible directive. There are no runtime Python dependencies. Documentation is built with Sphinx and published via Read the Docs.

Code Quality Testing is conformance-suite driven rather than unit-test-framework driven: scripts/run-tests.py runs the vendored json5-tests and JSONTestSuite corpora, and scripts/run-threaded-test.py stress-tests free-threaded builds for concurrency safety. Error handling is explicit and typed via a dedicated exception hierarchy (Json5DecoderException, Json5EncoderException, and specific subtypes like Json5NestingTooDeep, Json5IllegalCharacter, Json5ExtraData) rather than generic exceptions, with Cython’s except -1 propagation used throughout the C-level code paths. Public functions carry extensive numpydoc-style docstrings with Parameters/Raises/Returns sections and runnable doctested examples, and type stubs are shipped for static-typing consumers. CI runs the conformance suite across Ubuntu, macOS, and Windows on five Python versions per push.

What Makes It Unique The public API deliberately splits into narrowly-scoped functions per input/output medium (decode_buffer, decode_callback, decode_io, and their encode counterparts) instead of one function that inspects argument types, which makes zero-copy and streaming code paths explicit at the call site. A some=True flag lets callers decode multiple JSON5 documents sequentially from one stream — a capability the standard library’s json module lacks outright. The encoder’s ASCII-safe, HTML-injection-resistant output is called out as a deliberate security property rather than an incidental one. load/loads/dump/dumps are kept only as thin compatibility shims that nudge callers toward the richer native API rather than conflating the two.