pycparser

pycparser is a recursive-descent parser for C, written entirely in Python and closely following the grammar in Annex A of the C99 standard, with support for select C11 features. It has no external dependencies beyond a Python interpreter, which makes it trivial to vendor or install anywhere pip runs, including constrained CI and packaging environments.

The library exposes a hand-written lexer and parser (CLexer/CParser) that turn preprocessed C source into a typed AST (pycparser.c_ast), plus a CGenerator that can turn that AST back into C source. Because pycparser only needs to know whether a token is a previously declared typedef name (not its full semantics), it ships a set of minimal “fake” libc headers so real-world C files can be parsed without a target platform’s actual system headers.

pycparser is best known as the C-declaration parser inside cffi, where it powers automatic FFI generation from C function and type declarations, but it is also used to build C code obfuscators, static checkers, specialized C compiler front ends, and automatic unit-test discovery tools.

What You Get

• A recursive-descent CParser and standalone CLexer implementing the C99 grammar (Annex A), with partial C11 support
• A typed AST (pycparser.c_ast) covering declarations, expressions, statements, structs/unions/enums, and function definitions
• A CGenerator that renders an AST back into valid C source code, including a parenthesis-reduction mode
• Fake minimal libc headers (utils/fake_libc_include) so real-world #include-heavy C files parse without a target system’s actual headers
• A parse_file/preprocess_file convenience layer that shells out to cpp/gcc -E/clang -E for real-world preprocessing
• A suite of runnable examples (AST construction, AST rewriting, C-to-C translation, C-to-JSON serialization) under examples/

Common Use Cases

• Auto-generating FFI bindings from C headers, as cffi does internally
• Writing static analysis or linting tools that need a real C AST rather than regex/text scanning
• Building source-to-source C transformation tools, obfuscators, or instrumentation passes
• Extracting struct/function/type declarations from C headers for code generation (bindings, serializers, mock generators)
• Prototyping a specialized C compiler front end or a C dialect extension

Under The Hood

Architecture pycparser is organized as a small pipeline of standalone stages rather than a single monolithic parser. c_lexer.py defines a hand-written CLexer that tokenizes raw text (with callbacks for scope-entry/exit brace tracking and typedef lookup), c_parser.py’s CParser drives a recursive-descent parse over that token stream directly into the typed node tree defined in c_ast.py, and ast_transforms.py normalizes tricky constructs (atomic specifiers, switch-case bodies) before the AST reaches client code. A separate c_generator.py module walks the same AST with a visitor pattern to re-emit C source, keeping generation fully decoupled from parsing. The lexer’s type_lookup_func callback is the key coupling point: the parser injects live typedef-scope awareness into the lexer so that context-sensitive C grammar (distinguishing types from identifiers) can be resolved during a single top-down pass, without a separate semantic-analysis phase. Swapping the lexer or generator independently is explicitly supported (CParser(lexer=...)), so the core abstraction that would be costly to change is the AST node shape in _c_ast.cfg, which is code-generated into c_ast.py via _ast_gen.py.

Tech Stack The project targets Python 3.10+ with pyproject.toml/setuptools as the build backend and no runtime dependencies at all — a deliberate zero-dependency design confirmed in both the README and the packaging metadata. Development tooling is modern: ruff (pinned via uvx ruff@0.16.0) for linting and formatting, and ty (pinned via uvx ty@0.0.74) for static type checking, both invoked from a make check target, with uv.lock present for reproducible dev environments. CI (ci.yml) runs the full test suite across six Python versions (3.10 through 3.15, including prereleases) on Ubuntu, macOS, and Windows, giving strong cross-platform and cross-version coverage for a pure-Python parser that many other packages (like cffi) depend on transitively.

Code Quality The tests/ directory contains well over a hundred test functions split by concern — test_c_lexer.py, test_c_parser.py, test_c_ast.py, test_c_generator.py, test_general.py, and test_examples.py — run via python3 -m unittest discover, with fixture C files under tests/c_files. Core modules use modern Python idioms: @dataclass(slots=True) for the Token type, @dataclass for Coord, explicit type annotations (ClassVar, ParamSpec-style callables via collections.abc.Callable) throughout c_lexer.py, c_parser.py, and c_generator.py, and a dedicated ParseError exception rather than silent failures. The make check target enforces both lint and type-check cleanliness in CI, and the codebase carries decades of upstream scrutiny as a widely-depended-upon transitive dependency (notably via cffi).

What Makes It Unique pycparser’s distinguishing choice is doing full C grammar parsing in pure Python with zero external dependencies — no bundled C extension, no bindings to a system parser like libclang, and no code-generation step required at install time (the grammar tables are pre-generated and checked in). Its “fake libc headers” approach is a pragmatic, unusual design decision: rather than trying to parse a real system’s stdio.h and friends, it ships minimal stand-in headers that satisfy only the typedef-visibility requirement the parser actually needs, which keeps parsing fast and portable across platforms. This combination — pure-Python, dependency-free, AST-plus-generator, closely spec-following — is why it has become the de facto embedded C-declaration parser inside other widely used tools rather than a niche academic project.