sqlparse

sqlparse is a Python library for parsing, splitting, and formatting SQL statements without validating them against a specific database dialect’s grammar. It tokenizes raw SQL text into a navigable tree of statements, tokens, and identifiers, and exposes a format() function for reindenting, keyword-case normalization, and other cosmetic rewrites.

Because it doesn’t attempt to validate SQL correctness or target one specific database engine, sqlparse works across dialects and is widely embedded as a dependency by other tools - notably Django Debug Toolbar, ipython-sql, and various SQL linting/formatting utilities - as well as usable directly as a pre-commit hook for formatting SQL files before commit.

What You Get

• sqlparse.parse() producing a navigable token tree for any SQL statement
• sqlparse.split() for splitting multi-statement SQL strings
• sqlparse.format() with options for reindenting, keyword case, and comment stripping
• A pluggable lexer/filter pipeline (sqlparse.filters, sqlparse.engine) for custom formatting rules
• A sqlparse CLI entry point for formatting SQL files from the command line
• Usable directly as a pre-commit hook for automatic SQL formatting in CI

Common Use Cases

• Pretty-printing raw SQL queries in debugging tools like Django Debug Toolbar
• Splitting a file of multiple SQL statements before executing them one at a time
• Building custom SQL linters or formatters on top of the parsed token tree
• Normalizing keyword casing and indentation in a codebase’s SQL migration files
• Powering IDE or notebook tools (e.g. ipython-sql) that need to inspect SQL structure

Under The Hood

Architecture - sqlparse/lexer.py tokenizes raw SQL text using regex rules defined in keywords.py, sqlparse/engine groups tokens into a Statement tree via a pipeline of StatementFilter/grouping filters, sqlparse/sql.py defines the token/statement classes exposed to users, and sqlparse/filters implements the transformations (reindent, case normalization, comment stripping) that formatter.py composes into the public format() function.

Tech Stack - Pure Python with no runtime dependencies, targeting Python 3.8+, packaged with a standard pyproject.toml build, and shipping a py.typed marker for type-checker compatibility; cli.py/__main__.py provide a lightweight python -m sqlparse command-line entry point.

Code Quality - An extensive tests/ directory covers tokenization, grouping, formatting options, and CLI behavior; the project has a long history (created 2012, 1,363 total commits) with slowed but ongoing maintenance (moderate activity, ~2.3 commits/month), and BSD-3-Clause licensing with no unusual constraints.

API Design - The three-function surface (split, parse, format) covers the vast majority of use cases with minimal ceremony, and the token tree returned by parse() follows a consistent, well-documented class hierarchy (Statement -> Token/TokenList) that has remained stable enough for downstream tools like Django Debug Toolbar to depend on it for over a decade.