SQLGlot

SQLGlot is a pure-Python SQL parser, transpiler, optimizer, and lightweight execution engine with zero runtime dependencies. It reads SQL written for one dialect and can output syntactically and semantically correct SQL for another, supporting 31 dialects including DuckDB, Presto/Trino, Spark/Databricks, Snowflake, and BigQuery.

Beyond format conversion, SQLGlot exposes a full expression-tree API for programmatically building, analyzing, and rewriting SQL, plus a query optimizer and an in-process executor capable of running queries directly against Python objects. Its comprehensive test suite and dialect coverage have made it a common building block inside data-engineering tools that need to parse or translate SQL rather than hand-roll a parser.

What You Get

• A SQL parser and generator covering 31 dialects with cross-dialect transpilation
• A programmatic expression-tree API for building and rewriting SQL queries in code
• A query optimizer that normalizes, qualifies, and rewrites expression trees for better plans
• A pure-Python execution engine that can run SQL directly against in-memory data
• Column-level lineage and metadata analysis over parsed queries

Common Use Cases

• Translating SQL written for one warehouse (e.g. Snowflake) into another dialect (e.g. DuckDB or BigQuery)
• Statically analyzing or linting SQL queries inside a data pipeline before execution
• Programmatically generating SQL queries from application code via the expression-tree builder API
• Computing column-level lineage across a chain of SQL transformations for data governance

Under The Hood

Architecture - SQLGlot’s pipeline runs tokens.py/tokenizer_core.py to tokenize raw SQL, parser.py (a ~10k-line hand-written recursive-descent parser) to build a typed expression tree defined in sqlglot/expressions, then generator.py to re-render that tree back into dialect-specific SQL text; the sqlglot/optimizer package applies a pipeline of rule-based rewrites (qualification, simplification, pushdown) on the same tree, and sqlglot/executor walks it to evaluate queries directly against Python data structures without a real database.

Tech Stack - Pure Python (3.9+) with zero required runtime dependencies, packaged via setuptools with setuptools_scm for version derivation; dialect definitions live under sqlglot/dialects as per-database subclasses that override tokenizer/parser/generator behavior, keeping dialect-specific quirks isolated from the shared core.

Code Quality - The tests/ directory contains 61 test files exercising individual dialects, the optimizer, the executor, and diffing/lineage features; the project ships py.typed for downstream type-checking support and maintains a CHANGELOG documenting behavior changes across its frequent release cadence (100+ commits/month at the time of writing).

API Design - The top-level API is deliberately small — sqlglot.parse_one(), sqlglot.transpile(), and sqlglot.exp for tree construction cover the majority of use cases in one import, while power users can drop down to the expression tree, optimizer, or executor modules directly; this layering lets simple format-conversion tasks stay one-liners while still exposing the full internals for advanced query rewriting.