What You Need to Know
dbt on Databricks runs version-controlled SQL models through the dbt-databricks adapter, usually as a dbt task inside Lakeflow Jobs against a SQL warehouse. It earns its place when analysts own the silver and gold layers in SQL and you already have dbt skills. For streaming ingestion and change data capture, Lakeflow Declarative Pipelines is the stronger native choice.
- Cost: dbt Core is free; the SQL it generates runs on Databricks SQL warehouse compute, billed in DBUs.
- Effort: a working dbt task needs a Git folder, a serverless or pro SQL warehouse and a profile, which takes days, not weeks.
- Risk: running dbt and Declarative Pipelines on the same tables without a written split creates two sources of truth.
- Fit: keep dbt for SQL-first modelling teams; skip it for small teams that can do everything in Declarative Pipelines.
The transformation layer is where data becomes trustworthy or does not. A 2026 survey by Cloudera and Harvard Business Review Analytic Services found that only 7% of enterprises say their data is completely ready for AI. Much of the gap sits between raw tables and the curated models people actually query.
Failures in that layer are slow to fix. A Fivetran benchmark of large enterprises put the average time to resolve a pipeline failure at about 13 hours. When transformation logic is split across two frameworks with no clear owner, those hours go up.
So the dbt Databricks question is not whether the two work together. They do, and well. The question is whether a Databricks-first team should run dbt at all, or build silver and gold natively in Lakeflow Declarative Pipelines. Practitioners on the Databricks community forums still argue this one out, and there is no single right answer. There is a right answer for your team.
How dbt Runs on a Databricks Lakehouse
There are three moving parts. The first is the dbt Databricks adapter (package name dbt-databricks), which Databricks recommends over the older dbt-spark package; the adapter is a fork of dbt-spark tuned for Databricks. The second is compute. According to the Databricks documentation, the SQL that dbt generates runs on a serverless or pro SQL warehouse, and job compute cannot be the target.
The third part is orchestration. Lakeflow Jobs (formerly Databricks Workflows) has a native dbt task that runs dbt Core projects from a Git folder, with commands such as dbt deps, dbt seed and dbt run. You cannot run a project from DBFS. If you deploy with Declarative Automation Bundles (the March 2026 name for Databricks Asset Bundles), the bundle defines the job that carries the dbt task, so the whole thing lives in version control.
Python models are the exception worth knowing. The Databricks docs describe dbt Python model support in the dbt task as beta, and those models cannot run on a SQL warehouse. If a team’s models are mostly PySpark, dbt is the wrong wrapper.
What the Adapter Can Materialize
The dbt-databricks configuration reference lists tables, views, incremental models and snapshots, plus two materializations that matter a great deal on a lakehouse: materialized_view and streaming_table. Both create the same pipeline-backed objects that Lakeflow Declarative Pipelines manages, so a dbt project can declare incrementally refreshed tables without hand-written refresh logic.
Incremental models default to a merge on a unique key. Append, insert_overwrite, replace_where and microbatch strategies are also available for Delta tables. That covers most silver-layer patterns: deduplicating a conformed customer table, rolling up daily orders, or reprocessing a single late-arriving partition.
dbt vs Lakeflow Declarative Pipelines, Side by Side
| Decision factor | dbt on Databricks | Lakeflow Declarative Pipelines |
|---|---|---|
| Primary language | SQL with Jinja templating | SQL or Python |
| Best layer | Silver and gold business models | Bronze ingestion through silver, plus streaming |
| Change data capture | Incremental merge on a unique key | AUTO CDC with SEQUENCE BY for out-of-order records |
| Data quality | dbt tests after the model builds | Expectations enforced as rows flow |
| Compute | Serverless or pro SQL warehouse | Pipeline compute, serverless available |
| Portability | Project runs on other warehouses with a new adapter | Open core in Apache Spark Declarative Pipelines |
| Who owns it | Analytics engineers and analysts | Data engineers |
The portability row deserves a note. Databricks contributed its declarative pipeline framework to Apache Spark as Spark Declarative Pipelines, and the Lakeflow product extends it. Pipeline code is therefore less locked in than it used to be, which weakens the old argument that dbt is the only portable option.
Where Running Both Goes Wrong
The usual failure is not technical, it is ownership. A team adopts Lakeflow Declarative Pipelines for ingestion because AUTO CDC saves weeks of merge code, then keeps its existing dbt project for reporting models because the analysts know it. Nobody writes down where one stops and the other starts. Six months later a silver customer table exists in both, built with slightly different deduplication rules, and two dashboards disagree on customer count. Each tool’s lineage view looks correct on its own. The defect only shows up when a finance lead puts the two numbers side by side. Add the scheduling split, where dbt runs on a cron and the pipeline runs on a file-arrival trigger, and you get gold tables built from silver data that has not finished refreshing. In Lean Six Sigma language this is a handoff defect: the process has two owners and no control point between them. The fix is a one-page contract that assigns each medallion layer to exactly one framework, puts both under Lakeflow Jobs with explicit task dependencies, and registers everything in Unity Catalog so lineage crosses the boundary.
When dbt Earns Its Place on the Lakehouse
Keep or adopt dbt on a Databricks lakehouse when most of these hold:
- Analysts or analytics engineers write the business logic, and they think in SQL, not Python.
- You already have a dbt project with tests and documentation that people rely on.
- You want the same modelling code to run against more than one platform during a migration.
- The gold layer is a set of dimensional models for Power BI, where dbt tests and docs pay for themselves.
Skip it when the team is three data engineers doing everything, when most logic is streaming, or when change data capture from ERP databases dominates the workload. In those cases Declarative Pipelines does the job with one less framework to patch. For the orchestration side of that call, our data orchestration guide walks through the options.
One commercial fact to track: in October 2025 dbt Labs and Fivetran announced a merger. dbt Labs said dbt Core and its Fusion engine will keep shipping under their current licenses. Treat that as a reason to watch the roadmap, not a reason to rip anything out.
What to Do This Quarter, in This Order
Start by writing the layer contract. List every silver and gold table, name the framework that builds it, and delete or retire any table built twice. That single page prevents most of the disagreements described above, and it usually takes an afternoon with the people who own the dashboards. Next, move ingestion and change data capture into Lakeflow Declarative Pipelines if they are not already there, using AUTO CDC so nobody maintains hand-written merge statements; our Lakeflow guide covers the mechanics. Third, bring the dbt project under Lakeflow Jobs as a dbt task with explicit dependencies on the pipeline that feeds it, running on a serverless or pro SQL warehouse, and deploy both from one repository so a change to either shows up in the same pull request. Fourth, switch heavy models to incremental or materialized_view materializations and measure warehouse cost before and after. Finally, put pipeline and model failures on a weekly control chart. If the failure rate is not falling after a month, the process behind the data is the problem, not the tool. Analytics AIML helps teams draw that layer contract and build the pipelines behind it.
Frequently Asked Questions (FAQs)
Does dbt work with Databricks?
Yes. The dbt-databricks adapter connects dbt to Databricks SQL warehouses, and Lakeflow Jobs has a native dbt task that runs dbt Core projects from a Git folder. Databricks recommends dbt-databricks over the older dbt-spark package.
Should I use dbt or Lakeflow Declarative Pipelines?
Use Declarative Pipelines for ingestion, streaming and change data capture, and use dbt for SQL business models when analysts own them. If one small team owns everything, Declarative Pipelines alone is simpler. Whatever you choose, assign each table to exactly one framework.
Can dbt create streaming tables and materialized views on Databricks?
Yes. The dbt-databricks adapter supports materialized_view and streaming_table materializations alongside tables, views, incremental models and snapshots, which lets a dbt project declare incrementally refreshed objects on the lakehouse.