tree-sitter-lua

tree-sitter-lua is a formal grammar for the Lua language built on the tree-sitter parsing framework. It compiles Lua 5.x and LuaJIT source into a concrete syntax tree that stays incrementally up to date as the source changes, which is what lets editors and language tooling re-parse a file on every keystroke without re-scanning the whole thing. The grammar is expressed declaratively in grammar.js, backed by a small hand-written external scanner (src/scanner.c) that handles Lua’s context-sensitive long-bracket strings and comments ([[...]], [==[...]==]), a construct that a pure context-free grammar can’t express cleanly.

Beyond the raw parser, the project ships the queries that make a parse tree useful in practice: syntax-highlighting queries, local-variable scope queries, symbol tags for outline/go-to-definition, and injection queries for embedded languages. It publishes official bindings for C, Rust, Node.js, Python, Go, and Swift from a single grammar source, so the same battle-tested Lua grammar backs editor plugins, static-analysis tools, and embedding hosts across very different language ecosystems.

What You Get

• A complete Lua grammar (grammar.js) covering Lua 5.x and LuaJIT syntax, compiled to a generated C parser (src/parser.c)
• An external scanner (src/scanner.c) that correctly handles Lua’s nested long-bracket strings and comments
• Prebuilt query files for syntax highlighting, local-variable scoping, symbol tags, and language injections (queries/*.scm)
• Official language bindings for Rust, C, Node.js, Python, Go, and Swift generated from the same grammar source
• A large corpus test suite (test/corpus/*.txt) plus CI validation against real-world Lua code from Neovim, Kong, APISIX, and the reference Lua test suite

Common Use Cases

• Adding accurate Lua syntax highlighting to an editor or IDE via the exported HIGHLIGHTS_QUERY
• Powering incremental re-parsing in an editor so large Lua files stay responsive while typing
• Building static-analysis or linting tools that need a real Lua AST instead of regex-based parsing
• Generating symbol outlines or go-to-definition support from the TAGS_QUERY
• Embedding a Lua parser in a Rust, Node.js, Python, Go, or Swift application via the matching language binding

Under The Hood

Architecture The grammar is defined once in grammar.js as a declarative rule set (statements, expressions, declarations, variables as tree-sitter supertypes) with an explicit operator-precedence table, then compiled by the tree-sitter CLI into a generated parser (src/parser.c, ~14k lines) plus node-types.json and grammar.json metadata. A small external scanner (src/scanner.c) plugs into the generated parser for the one piece of Lua syntax that isn’t context-free: matching nested [==[...]==] long-bracket delimiters for strings and comments, tracked via a tiny serialized Scanner struct (ending_char, level_count) so the state survives incremental re-parses. Six thin language bindings (bindings/rust, node, python via setup.py, go, swift via Package.swift, and the raw C API) each wrap the same compiled grammar behind idiomatic per-language entry points, so changing the grammar only ever means regenerating one artifact.

Tech Stack The grammar source is authored in JavaScript against the tree-sitter-cli DSL and compiled to C; the Rust crate wraps it via tree-sitter-language 0.1 with a build.rs that invokes cc 1.2 to compile parser.c and scanner.c, and depends on tree-sitter 0.26 only as a dev-dependency for its own tests. Parallel packaging exists for npm (binding.gyp/node-gyp), PyPI (setup.py/pyproject.toml), Go (go.mod), and Swift Package Manager (Package.swift), all built from the same tree-sitter.json grammar manifest. CI runs the matrix across Ubuntu, Windows, and macOS using the official tree-sitter/setup-action and parser-test-action.

Code Quality Correctness is validated through tree-sitter’s corpus-test format: over 1,500 lines across four corpus files (chunk.txt, comments.txt, expressions.txt, statements.txt) pair Lua source snippets with their expected S-expression parse trees, plus dedicated highlight and tags fixture tests. CI goes further than most grammars by cloning real-world Lua codebases at pinned refs (Neovim, Kong, APISIX, luvit, and the upstream Lua reference test suite) and parsing them as a fuzz-style regression check, in addition to the corpus suite, across three operating systems. The Rust binding adds its own doctest and a unit test asserting the grammar loads. There’s no traditional application error-handling to assess since this is a parser/scanner pair, but the scanner itself is small, single-purpose, and easy to audit.

API Design Consumer-facing surface area is deliberately tiny: each binding exposes one language constant (e.g. Rust’s LANGUAGE) plus static string constants for the highlighting, locals, tags, and injection queries, so wiring up a working parser is a 3-line snippet in any supported language. That minimalism is consistent across all six bindings, meaning the learning curve for one language transfers directly to the others, and it mirrors the conventions of the broader tree-sitter grammar ecosystem rather than inventing its own.