lit

lit (the LLVM Integrated Tester) is a portable command-line tool for executing LLVM- and Clang-style test suites, summarizing their results, and clearly indicating failures. It is designed to be lightweight with as simple a user interface as possible, while scaling to the tens of thousands of tests that make up the LLVM project.

Though it originated inside LLVM, lit is a general-purpose test runner distributed on PyPI and used well beyond compilers. It discovers tests flexibly, runs them in parallel, supports multiple test formats, and drives the ubiquitous “RUN:” line style of shell-based tests, making it a practical harness for any project that wants file-driven, tool-oriented testing.

What You Get

• A portable lit command-line runner installable from PyPI
• Flexible, configuration-driven test discovery via lit.cfg files
• Parallel test execution across multiple worker processes
• Support for shell-style RUN: line tests and multiple custom test formats
• Concise result summaries with clear indication of failures and timing data

Common Use Cases

• Running LLVM/Clang-style regression suites for a compiler or toolchain project
• Executing file-driven tests that embed shell commands as RUN: lines
• Parallelizing large test suites for faster CI feedback
• Building a custom test harness by supplying project-specific lit configuration

Under The Hood

Architecture - lit’s flow starts in main.py, which parses CLI arguments (cl_arguments.py) and drives discovery.py to walk the given paths and load TestingConfig objects from per-directory lit.cfg files. Discovered Test objects are dispatched through run.py to a pool of worker.py processes; each test is executed by a format (in formats/), most commonly the shell-based runner in TestRunner.py which interprets RUN: lines using the internal shell utilities (ShUtil.py, ShCommands.py, ShellEnvironment.py). Results flow back through display.py/ProgressBar.py and reports.py.

Tech Stack - Pure Python packaged with setuptools (pyproject.toml, setup.py), distributed on PyPI as lit. It has essentially no third-party runtime dependencies, relying on the standard library and multiprocessing for parallelism, which keeps it portable across platforms.

Code Quality - As LLVM’s core regression harness the tool is extremely mature and self-hosting: its own tests/ directory (23+ files with test logic) uses lit to test lit. A built-in BooleanExpression engine handles feature/XFAIL logic, and the module boundaries between discovery, execution, and reporting are clear.

API Design - The primary interface is the lit command plus the lit.cfg/lit.local.cfg configuration model. For simple use the CLI is minimal (lit path/), while advanced projects customize discovery, substitutions, and formats through well-documented config hooks; the man page and LLVM Command Guide provide thorough reference material.