dbt-databricks

dbt-databricks is the official dbt adapter that connects dbt Core to the Databricks Lakehouse Platform. Built on the foundation of dbt-spark, it contains the connection management, credential handling, and materialization logic dbt needs to compile and run models, tests, snapshots, and seeds against Databricks SQL warehouses and clusters.

Maintained by Databricks, it adds Databricks-native capabilities such as Unity Catalog support, Delta Lake materializations, and authentication through the Databricks SDK and SQL connector. Installing it alongside dbt Core is all that is required to run an analytics-engineering project on Databricks.

What You Get

• A dbt adapter that targets the Databricks Lakehouse as the execution engine for dbt models
• Databricks connection and authentication via the Databricks SDK and SQL connector
• Unity Catalog support for three-level namespaces and governance
• Delta Lake materializations including incremental and merge strategies
• Integration with the standard dbt Core CLI workflow (run, test, build, snapshot)

Common Use Cases

• Running a dbt analytics-engineering project against Databricks SQL warehouses or clusters
• Building Delta Lake incremental and merge transformations with dbt
• Managing Databricks credentials and Unity Catalog targets through dbt profiles

Under The Hood

Architecture - The adapter lives under dbt/adapters/databricks/. connections.py and handle.py manage Databricks SQL/cluster connections, credentials.py handles the token/OAuth authentication flows, relation.py and parse_model.py map dbt relations onto Unity Catalog’s three-level namespace, and logging.py/global_state.py provide cross-cutting infrastructure. It extends the dbt-spark adapter, inheriting Spark SQL behavior and layering Databricks-specific Delta and catalog logic on top.

Tech Stack - Python packaged with Hatch. Key runtime dependencies include databricks-sql-connector and databricks-sdk for connectivity, dbt-spark, dbt-adapters, dbt-common, and dbt-core for the plugin contract, plus keyring, pydantic, and click.

Code Quality - The repository runs both unit and integration test workflows in CI (advertised via README badges) and maintains an extensive tests/ tree alongside a detailed changelog. Being Databricks-maintained, it is very actively developed with a strong maintenance cadence, reflected in its high repo-health score.

API Design - Users configure Databricks through dbt YAML profiles rather than the Python API, specifying host, HTTP path, and auth. Because it follows the standard dbt adapter contract and extends dbt-spark, the experience is familiar to existing dbt users, with Unity Catalog and Delta features exposed through model configs.