jsonpath-ng

jsonpath-ng is a full JSONPath language implementation for Python, not just a regex-based matcher. It merges the earlier jsonpath-rw and jsonpath-rw-ext projects into a single library, so expressions are first-class parsed objects that can be inspected, transformed, printed, and extended rather than opaque strings.

Beyond simple find() lookups, the library supports updating and removing matched nodes in place, filtering data by predicate, automatic ID generation for unlabeled records, and an extended syntax (via jsonpath_ng.ext) that adds arithmetic, filter, string, and sort operators on top of the base grammar. A bundled jsonpath_ng CLI script lets you run JSONPath queries against JSON files or stdin without writing any Python.

What You Get

• Standard JSONPath parsing - an LALR parser (built on PLY) turns JSONPath strings into a typed AST instead of matching with fragile regexes.
• Extended operator syntax - jsonpath_ng.ext adds arithmetic, comparison (==, !=, <, >, <=, >=, =~), boolean filters, sort direction, and string functions like `len` and `keys`.
• In-place mutation - update(), update_or_create(), and filter() let you rewrite or strip matched values directly in the source data structure.
• Path-aware matches - every DatumInContext result tracks its full_path, so you can print, compare, or reconstruct the exact path that produced a value.
• Automatic ID assignment - setting jsonpath.auto_id_field backfills a chosen field with a generated JSONPath-based ID for any record missing one.
• Bundled CLI - the jsonpath_ng console script runs a JSONPath expression against files or stdin from the shell, no Python glue code required.

Common Use Cases

• Extracting nested API fields - a service pulls specific values out of deeply nested third-party JSON responses without writing manual key-chasing code.
• Config and manifest querying - tooling that needs to read or validate specific paths inside JSON/YAML-derived config trees uses jsonpath-ng instead of ad hoc dict traversal.
• Bulk transforming JSON documents - a data pipeline uses update()/filter() to redact, normalize, or strip fields across many documents matching one path expression.
• Building JSONPath-driven query tools - applications that expose a JSONPath query box to end users (e.g. API explorers, log viewers) embed jsonpath-ng as the execution engine.

Under The Hood

Architecture jsonpath-ng is organized as a small compiler pipeline: lexer.py tokenizes a JSONPath string using a PLY-based JsonPathLexer, parser.py’s JsonPathParser turns the token stream into an AST of JSONPath subclasses defined in jsonpath.py (Root, This, Fields, Slice, Index, Child, Descendants, Where, Union, and more), and each node implements a shared interface (find, update, filter, child) so any AST node can be evaluated against arbitrary JSON data or composed into a larger expression. Results are wrapped in DatumInContext, which threads the traversal path back through nested calls so full_path can be reconstructed after the fact. The jsonpath_ng.ext package layers an ExtendedJsonPathLexer/ExtendedJsonPathParser on top of the base grammar, adding arithmetic, filter, iterable, and string operator modules without modifying the core AST classes, and jsonpath_ng/bin/jsonpath.py provides a thin CLI wrapper around parse() and find().

Tech Stack The library is pure Python (3.10+ per CI and classifiers) with a single runtime dependency family: a vendored/forked _ply package (Python Lex-Yacc) used for lexing and LALR parsing, rather than a hand-rolled regex matcher. Packaging is done with classic setuptools (setup.py), tests run under pytest with tox orchestrating the version matrix, and CI (.github/workflows/ci.yml) exercises Python 3.10 through 3.14 (including prereleases) on every push and PR via GitHub Actions, plus a separate CodeQL workflow for static security scanning.

Code Quality The tests/ directory contains dedicated suites for the lexer, parser, core JSONPath semantics, the jsonpath-rw-ext extension surface, creation/mutation behavior, exceptions, and round-tripping expressions back to strings — over a thousand lines of tests total, run with pytest and configured in pyproject.toml to escalate warnings to errors (with a narrow exception for a known _ply ResourceWarning). Code style favors explicit AST classes over generic dict/regex hacks, and JsonPathLexerError/JsonPathParserError give typed, specific error paths for malformed expressions rather than silently swallowing them. There’s no separate linter/formatter config visible in the repo, and the multi-version CI matrix is the main quality gate beyond the test suite itself.

What Makes It Unique Unlike simpler dict-traversal or regex-based JSONPath shims, jsonpath-ng treats JSONPath as a real grammar with a proper lexer/parser and a fully composable AST — expressions are objects you can build programmatically (Fields('foo').child(Slice('*'))), inspect, and extend, not opaque strings evaluated by a black box. Its merge of jsonpath-rw and jsonpath-rw-ext into one project, plus support for in-place update/filter mutation and full-path-aware results, goes beyond what most other Python JSONPath implementations offer, which are typically read-only.