jmespath.py

jmespath is the reference Python implementation of JMESPath, a declarative query language purpose-built for extracting, filtering, and reshaping data inside JSON documents. Instead of writing manual loops and conditionals to dig values out of nested dicts and lists, you write a single expression such as foo.bar[*].name or people[?age > 30].name and evaluate it against any Python data structure with jmespath.search().

Under the hood the library is a from-scratch Pratt (top-down operator precedence) parser: an expression is tokenized, parsed into an AST, and then walked by a tree interpreter that implements the full JMESPath specification (projections, flattening, filters, pipes, multi-select lists/hashes, and a standard function library). Expressions can be pre-compiled with jmespath.compile() for repeated searches, and the library supports custom functions via a small metaclass-based registration system — the same engine that powers the --query flag in the AWS CLI and botocore.

What You Get

• A two-function public API (search and compile) modeled on Python’s re module for parsing once and reusing an expression across many searches
• Full JMESPath language coverage: field/subexpressions, wildcards, projections, flatten, filters, pipes, multi-select lists and hashes, and comparison/boolean operators
• A built-in function library (length, sort_by, merge, contains, type, etc.) plus a functions.signature-decorated extension point for registering custom functions
• An Options object for controlling evaluation, including a dict_cls override for predictable (e.g. OrderedDict) key ordering in multi-select-hash results
• A standalone jp.py command-line tool for querying JSON files or stdin, with an --ast flag to inspect the parsed expression tree

Common Use Cases

• Extracting and reshaping fields from JSON API responses in application code
• Powering the --query flag in the AWS CLI and botocore-based tooling
• Filtering and transforming JSON documents mid-pipeline in ETL/data-processing scripts
• Writing ad-hoc JSON queries against files or stdin from the command line via jp.py
• Asserting or extracting values from nested JSON configuration in CI/validation scripts

Under The Hood

Architecture jmespath.py is a classic two-stage interpreter: lexer.py tokenizes an expression, parser.py implements a top-down operator precedence (Pratt) parser that dispatches on token type via _token_nud_*/_token_led_* methods and a BINDING_POWER table to get operator precedence right, and visitor.py’s TreeInterpreter (a Visitor subclass with method-name dispatch and a per-instance method cache) walks the resulting dict-based AST built by ast.py to produce a result. The public surface in jmespath/__init__.py is intentionally tiny — compile() and search() — with a class-level LRU-ish cache (Parser._CACHE, capped at 512 entries) so repeated searches of the same expression skip re-parsing. Because AST nodes are plain dicts keyed by type/children/value, the AST shape is an implicit contract shared across ast.py, visitor.py, and the parser’s led/nud methods — changing it means touching all three in lockstep.

Tech Stack The library itself has zero runtime dependencies and targets Python 3.9 through 3.14 (per setup.py’s python_requires and the GitHub Actions test matrix across Ubuntu, macOS, and Windows). Development/test dependencies (requirements.txt) are pytest 8.4.1 with pytest-cov, and Hypothesis for property-based testing, plus conditional setuptools/packaging pins for Python 3.12+. Packaging goes through plain setuptools/find_packages, and the project ships a small stdlib-only CLI (bin/jp.py, using argparse and json) alongside the library.

Code Quality Tests are organized by concern — test_lexer.py, test_parser.py, test_functions.py, test_custom_functions.py, test_search.py — plus a data-driven test_compliance.py that runs the shared cross-language JMESPath compliance suite from tests/compliance/*.json (covering wildcards, filters, slices, pipes, unicode, and more). CI runs this suite with coverage reporting across the full OS/Python matrix, and a separate CodeQL workflow performs static security analysis. Errors are surfaced through an explicit exception hierarchy (ParseError, LexerError, JMESPathTypeError, ArityError, etc.) in exceptions.py rather than being swallowed. The codebase has no static type hints or mypy configuration, relying instead on runtime type-checking against a Python-type-to-JMESPath-type map in functions.py.

API Design The public API deliberately mirrors Python’s re module (search for one-off use, compile for reuse), which makes it immediately familiar to any Python developer. Extensibility is handled through a FunctionRegistry metaclass that auto-discovers any _func_* method decorated with @functions.signature(...) on a Functions subclass, so adding a custom JMESPath function requires no manual registration step — just a documented pattern. The tradeoff is sparse in-code documentation: the README and the external jmespath.org specification carry most of the explanatory burden, rather than docstrings or type hints in the library itself.