@lezer/common

@lezer/common provides the core syntax-tree data structure and parser abstractions that every Lezer-based parser builds on. Lezer is an incremental parser system designed for editor use (it’s the parsing engine behind CodeMirror 6’s language support), and this package defines the memory-efficient, packed tree representation that parse results are stored in, along with the Parser/Input interfaces that concrete language parsers implement.

Rather than producing a conventional AST of allocated node objects, Lezer parsers built on this package output a compact, cursor-navigable tree that can be incrementally reused across edits — critical for keeping syntax highlighting and code folding fast in a live text editor as the user types.

What You Get

• The Tree type: a compact, packed syntax-tree representation with cursor-based traversal instead of individually allocated node objects
• TreeCursor and SyntaxNode for navigating and inspecting parsed trees
• The abstract Parser and Input interfaces that concrete Lezer parsers (e.g. @lezer/javascript, @lezer/python) implement
• NodeType/NodeSet and NodeProp for attaching typed metadata to grammar node kinds
• Tree-mixing (mix.ts) support for embedding one language’s parse tree inside another (e.g. JS inside HTML <script> tags)
• Incremental re-parsing support so edits to a document only reparse the changed region, not the whole file

Common Use Cases

• Building or consuming a Lezer-generated parser for a programming language inside an editor (most commonly CodeMirror 6)
• Implementing syntax highlighting, code folding, or bracket matching against a Lezer parse tree
• Embedding one language’s grammar inside another (e.g. CSS/JS inside HTML) via tree mixing
• Writing editor tooling that needs incremental reparsing to stay responsive on large documents
• Building static analysis or linting tools on top of an existing Lezer grammar’s tree output

Under The Hood

Architecture — The package is intentionally small and focused: tree.ts (~1,700 lines) implements the packed Tree/TreeBuffer/TreeCursor data structures at the core of the library, parse.ts defines the abstract Parser/PartialParser/Input interfaces concrete grammars implement, mix.ts implements the language-nesting/mixed-parsing machinery (e.g. embedding JS inside HTML), and index.ts just re-exports the public surface. Concrete language parsers (generated by the separate @lezer/generator from a grammar file) depend on this package for the tree/parser abstractions rather than reimplementing tree storage themselves — which is what keeps every Lezer-based language parser interoperable and lets CodeMirror 6 treat them uniformly.

Tech Stack — Pure TypeScript, compiled to both ESM (dist/index.js) and CJS (dist/index.cjs) via a small custom build.js using @marijn/buildtool. It has zero runtime dependencies — the packed tree format is implemented from scratch rather than wrapping an existing parsing library, which keeps it lightweight enough to load in a browser-based editor.

Code Quality — Tests live under test/test-tree.ts, run with Mocha, focused specifically on tree construction/navigation correctness (the highest-risk area given the packed binary-like tree representation). Primary development for this and Marijn Haverbeke’s other editor projects (CodeMirror, Lezer) now happens on his self-hosted Forgejo instance at code.haverbeke.berlin — the GitHub repository is a kept-in-sync mirror (commits and npm releases continue to land here), not an abandoned copy, but issues/PRs are handled upstream rather than on GitHub.

API Design — The public surface centers on a handful of well-documented types (Tree, TreeCursor, SyntaxNode, NodeType) with consistent cursor-based traversal patterns across all of them, and the canonical reference docs live on the Lezer website rather than in inline comments, so newcomers are expected to read the docs site rather than infer usage from types alone. Because this package is rarely used standalone — it’s almost always pulled in transitively by a concrete language parser or CodeMirror itself — most developers interact with a thin slice of its API (walking a SyntaxNode tree) rather than the full interface surface.