ClickHouse + dbt: Building a GitOps workflow at Twilio

At a dbt Summit 2026 breakout session titled "ClickHouse + dbt: Building a GitOps workflow at Twilio," Twilio's Voice & Video Insights team, Marek Radonsky and Pavan Kotikalapudi, walked through how they replaced three separate distributed systems with a single ClickHouse-backed pipeline governed by dbt, joined by Alex Francoeur, director of product management at ClickHouse.

Twilio's Voice Insights product gives customers near real-time analytics on every call and conference running through its platform, and the company processes more than 2.5 trillion interactions a year for customers such as Best Buy, Netflix, and Uber, according to Twilio.

For a decade, that pipeline ran on three separate distributed systems: Kafka for ingesting raw call metrics, Spark Streaming applications to process the data, and Elasticsearch to serve results to Twilio's console and REST APIs.

Running three enterprise-scale systems for one workflow created a heavy operational and infrastructure burden, and once schema changes had to scale across multiple developers and teams, making manual changes directly to the cluster became increasingly risky.

Twilio consolidated ingestion, transformation, and serving into ClickHouse alone, using materialized views to handle streaming transformations that used to require custom Spark Scala applications.

The team's workflow evolved in stages: from tracking DDL changes in shared docs, to version control that still allowed direct changes to the cluster, to a migration tool paired with a GitOps approach that stored schema as code in GitHub, and finally to dbt paired with the open-source dbt-clickhouse adapter for transformations, testing, and lineage.

As the number of pipelines grew, Twilio split a single monolithic dbt project into separate projects per product area with a shared layer for common logic, orchestrated through Argo Workflows, a Kubernetes-native orchestration engine similar to Airflow.

Because ClickHouse is open source, Twilio's engineers could spin up real ClickHouse test containers for local and CI integration tests rather than relying on mocks. Each developer can now also deploy their own preview environment against a slice of production data, an approach the team calls a database per branch.

Automated lineage generated from dbt artifacts replaced hand-maintained architecture diagrams, and Twilio contributed a materialized-view model update capability it needed back to the open-source dbt-clickhouse project.

The end result: three distributed systems became one, custom Spark applications became SQL models and materialized views in dbt, manual production changes became version-controlled and audited, and Twilio can now redeploy the same pipeline consistently across its regions.

Francoeur closed with product news: the long-established dbt v1 ClickHouse adapter remains fully supported, a dbt v2 adapter had just entered public beta and was nearing feature parity with v1, and ClickHouse support in the dbt platform was newly available in private beta.

Watch every keynote and breakout from dbt Summit 2026 on demand. Watch dbt Summit 2026 on demand →