lark

Lark is a parsing library for Python that lets you parse any context-free grammar using an EBNF-based grammar syntax, automatically building an annotated parse tree without writing manual AST construction code. It implements three parsing algorithms — Earley (with an SPPF forest for full ambiguity support), LALR(1) (fast, with a parse-aware contextual lexer), and CYK — so developers can trade expressive power against performance depending on grammar complexity.

Beyond the core parser, Lark provides tooling for real-world use: a stand-alone parser generator that emits a dependency-free LALR(1) parser file for embedding in other projects, grammar composition (importing terminals/rules across grammar files), a standard library of common terminals, and utilities like automatic reconstruction (regenerating text from a parse tree) and Nearley.js grammar import. It’s pure Python, fully typed with mypy, and used in production by projects like Poetry, Vyper, and Hypothesis.

What You Get

• Earley parser with SPPF forest support for handling any context-free grammar, including fully ambiguous ones
• LALR(1) parser with a contextual, parse-aware lexer for fast, linear-time parsing of unambiguous grammars
• Automatic parse-tree (AST) construction from the grammar alone — no manual tree-building code required
• A stand-alone parser generator that outputs a dependency-free LALR(1) parser file for embedding elsewhere
• Grammar composition, a standard terminal library, and Nearley.js grammar import for reusing existing grammars

Common Use Cases

• Building a domain-specific language (DSL) or configuration format parser without hand-writing a recursive-descent parser
• Parsing existing programming languages or file formats (e.g. JSON, SQL dialects, smart-contract languages) for tooling like linters, formatters, or interpreters
• Extracting structured data from semi-structured or ambiguous natural-language-like text using the Earley parser
• Embedding a small, dependency-free parser into a distributed package via the stand-alone parser generator

Under The Hood

Architecture Lark’s core sits in lark/lark.py, which owns the public Lark class — a thin facade wiring together grammar loading (load_grammar.py, handling EBNF parsing, imports, and templates), lexer construction (lexer.py: BasicLexer, ContextualLexer, LexerThread), and parser construction via parser_frontends.py’s dispatch logic, which routes to one of three implementations under lark/parsers/ (earley.py with an SPPF forest in earley_forest.py, lalr_parser.py wrapping a table-driven LALR(1) automaton from lalr_analysis.py, and cyk.py) — all conforming to a common frontend interface so the parsing strategy is swapped via an option string rather than subclassing. Parse output flows through parse_tree_builder.py into Tree/Token nodes, and visitors.py provides Visitor/Transformer/Interpreter dispatch classes for walking that tree, closing the loop between grammar text and usable AST. Grammar analysis, lexing, parsing algorithm, and tree construction are cleanly separated, swappable layers behind the single Lark() constructor.

Tech Stack Lark is pure Python with no required dependencies, targeting Python 3.8+, with optional extras for regex (Unicode character classes), nearley (js2py, for importing Nearley.js grammars), atomic_cache (atomicwrites, for safe grammar-cache writes), and interegular (regex-collision warnings). Packaging uses setuptools with setuptools-scm for git-tag-derived versioning, and the package ships py.typed for downstream type-checking. Its only deployment artifact beyond the library itself is the stand-alone parser generator, which emits self-contained Python source with zero runtime dependencies for embedding in other codebases.

Code Quality Tests live under tests/ across roughly twenty files (test_parser.py, test_lexer.py, test_grammar.py, test_cache.py, test_reconstructor.py, test_typing.py, and more), run via python -m tests, with CI spanning Python 3.8 through pre-release versions plus PyPy and Windows, a dedicated mypy workflow enforcing static typing, and a codecov workflow tracking coverage. Error handling uses a typed exception hierarchy rooted at LarkError with specific subclasses (GrammarError, ParseError, LexError, UnexpectedInput and its UnexpectedToken/UnexpectedCharacters/UnexpectedEOF children) carrying structured context such as line/column rather than raising bare or swallowed errors. Naming is consistent, the package is extensively type-annotated, and there’s no evidence of lint-suppressed or dead code in the core modules.

API Design Getting started is a single Lark(grammar_string) call — no scaffolding or builder pattern — with the grammar itself as plain-text EBNF, so the entire API surface for the common case is one class plus a .parse() method. Advanced usage layers on cleanly through keyword options rather than separate classes, and visitors.py offers an idiomatic Visitor/Transformer/Interpreter dispatch pattern for post-processing trees. Documentation is extensive: a full hosted docs site, a JSON-parsing tutorial, a grammar reference, a features list, and abundant runnable examples covering standalone generation, grammar composition, and reconstruction. Offering Earley, LALR(1), and CYK behind one uniform, swappable-lexer interface is a genuinely distinctive design choice among Python parsing libraries, most of which implement only one algorithm.