Automating the impossible: migrating 40,000+ objects to dbt in 9 months

This talk shares our end-to-end automation strategy making the impossible possible. We built a system handling environment setup with production scoped parallels, SQL conversion to dbt ready code, dependency mapping, and full testing with results publication in Snowflake dashboards. The result: over 40 well-managed data products delivered on time. 

But automation wasn’t the whole story. Ensuring data quality during an active migration was the hardest challenge. Nondeterministic models, duplicate data, and adopting proper CI/CD were only some of the issues solved to keep business continuity. 

This session offers a practical, battle-tested playbook for large-scale migrations: when to automate and when a manual work is non-negotiable, how to scale testing, and how to manage organizational change.