sqlfluff

SQLFluff is a configurable SQL linter and auto-formatter built for ELT and analytics engineering workflows. Rather than assuming a single SQL flavor, it ships dialect definitions for dozens of databases and query engines, from Postgres and Snowflake to niche targets like SOQL and Teradata, and can parse SQL wrapped in Jinja or dbt templating so linting rules apply to the rendered query rather than choking on template syntax.

Run as a CLI (sqlfluff lint / sqlfluff fix) or via its Python API, it reports precise rule violations with line/position references and can auto-fix the majority of layout and style issues in place, making it a common addition to pre-commit hooks and CI pipelines for teams standardizing SQL style across a codebase or dbt project.

What You Get

• A sqlfluff lint / sqlfluff fix CLI that reports and auto-corrects style violations directly against .sql files
• Dialect grammars for 30+ SQL flavors (ANSI, Postgres, BigQuery, Snowflake, T-SQL, Redshift, and more) selectable via a single --dialect flag
• Built-in support for Jinja, dbt, and Python-format-string templating so rules run against rendered SQL, not raw template text
• A pluggy-based plugin system (sqlfluff.core.plugin) for adding custom rules, dialects, or templaters without forking the project
• A public Python API (sqlfluff.api.simple) for calling lint/fix programmatically from scripts or other tooling
• An in-progress Rust-accelerated parser (sqlfluffrs) that acts as a drop-in replacement for the Python parser on large codebases

Common Use Cases

• Enforcing a consistent SQL style guide across a team via a pre-commit hook
• Linting and auto-fixing .sql models inside a dbt project, including templated Jinja blocks
• Gating SQL style in CI so pull requests fail on unformatted or non-conforming queries
• Migrating a codebase between SQL dialects and using dialect-specific rules to catch incompatible syntax

Under The Hood

Architecture SQLFluff separates concerns into a core layer (config, templaters, lexer, parser, linter, plugin host) and thin outer layers (api, cli, dialects, rules) that depend on it but never the reverse, a boundary enforced mechanically via importlinter contracts in pyproject.toml (e.g. “Forbid dependencies outside core”, “API may not depend on CLI”). Linting flows from core.templaters (renders Jinja/dbt/placeholder syntax into plain SQL plus a source map), through core.parser (a recursive grammar-driven matcher producing a BaseSegment tree), into core.linter, which walks the tree applying the enabled rules and can rewrite segments in place for auto-fix. Dialects extend the base ANSI grammar via subclassing rather than duplicating grammars, and a pluggy-based plugin host (core/plugin) lets external packages register additional rules, dialects, or templaters through named entry points without modifying core.

Tech Stack The project targets Python 3.10+ and is built on click for the CLI, Jinja2 and pyyaml/tomli for config and templating, regex for pattern matching beyond stdlib re, tblib for cross-process exception propagation under multiprocessing, and pluggy for its plugin architecture; performance-sensitive parsing is being reimplemented in a companion Rust crate (sqlfluffrs, versioned and released in lockstep with the Python package) that plugs in as a drop-in parser backend. Packaging uses setuptools with a standard pyproject.toml, and the project is also published as a Docker image and distributed dbt/sqlmesh integration packages.

Code Quality The repository carries an extensive test suite (159+ test files spanning test/core, test/dialects, test/rules, test/cli, and test/api) exercising dialect parsing fixtures, rule behavior, and CLI output; typing is enforced via mypy with per-module overrides, and ruff handles both linting and formatting (line length 88, isort ordering, pydocstyle conventions) as a pre-commit and CI gate. Architectural boundaries are additionally enforced by importlinter contracts that fail CI if core starts depending on outer layers, an unusually rigorous check for a project this size. Multiple GitHub Actions workflows (ci-tests.yml, ci-test-dbt.yml, ci-test-sqlmesh.yml, pre-commit.yml) run the suite against dbt and sqlmesh integrations in addition to the core package.

API Design The public surface is deliberately layered: a full-featured CLI (sqlfluff lint|fix|parse|format) for everyday use, plus a narrower sqlfluff.api.simple module (lint, fix, get_simple_config) for embedding in other Python tools without needing to construct FluffConfig/Linter objects directly. Configuration is resolved through a predictable cascade (CLI overrides > .sqlfluff files > defaults), and the plugin hook specification (core/plugin/hookspecs.py) gives third-party dialect/rule authors a documented, versioned contract to build against rather than reaching into internals.