dbt Summit

All aboard: Rebuilding dbt in the open

Thursday, September 171:45 PM PT
Level 3, Gracia 4

Rewriting a 9 year old code base is hard. Rewriting it in the open over ~1 billion dbt installs later? Harder. Why did we do it? For faster runtimes, SQL comprehension, shared adapters, better docs, and a modernized developer experience.

We'll be rooting our session in a metaphor from a similar engineering problem, but for trains back in the 1800s: the break of gauge. I'll teach you how we aren't going to pay the cost of transfer as we modernize onto OSS v2.

This session will be demo packed:

  • v1 → v2 conversion using autofix, and the v2 parser already running inside v1.12
  • Side by side parse times in v1 and v2
  • The power of one binary shipped with every adapter
  • Docs v2 because we both know your `manifest.json` is too big
  • Time allowing: how the distributions differ at catching an error (spoiler: it's 1.68s vs. 0.04s — wouldn't you like to know how to catch an error 42x faster?)

By the end of this session you should know:

  1. Why you would want to upgrade to v2 and how to use the flagship v2 upgrades
  2. How to upgrade to v2 — a project upgrade will be shown
  3. How to choose the right distribution for you
  4. More of the v2 mechanics and how it works under the hood

 

I hope to see you there!

Check out more sessions

View all sessions
dbt Summit on stage

Ready to join us?

Join data leaders and practitioners at dbt Summit for three days of ideas, skills, and shared progress.