Rebuilding dbt Core in the open: Faster runtime, adapters, and docs v2
dbt v2 bundles adapters into a single binary, parses large projects several times faster, and replaces the sprawling manifest.json with small, queryable files built for both people and AI agents.
Overview
Covers dbt Core v2.0's redesign: a Rust-based engine shared with dbt v2, improved adapter architecture via ADBC and Arrow, and parquet-backed artifacts for better docs on large projects. Covers the shared engine/contribution path, changes for maintainers, migration from v1.x, and performance comparisons.
What you'll take away
- 01
dbt v2 gives dbt real semantic comprehension of SQL and Jinja for the first time, by rewriting the templating engine in Rust, rather than just orchestrating a DAG without understanding the SQL inside it.
- 02
On a synthetic 1,800-model project, v1 parsed in 10-13 seconds versus about 3 for v2, and the new linter runs 50 to 100 times faster than standard SQLFluff.
- 03
Docs v2 replaces a single, half-a-million-line manifest.json with small, queryable Parquet files, enabling plain-language lineage and "blast radius" queries that used to mean manually parsing a massive file.
Session speaker
Hope Watson
dbt Labs