Does your data pipeline deserve to exist? How EF Education built a cost and value layer with dbt and Cortex Agent
EF Education First built a Snowflake Cortex Agent that scores every dbt model on cost and value, cutting some pipeline runtimes by 80 percent.
Overview
Most teams can monitor pipeline execution but few can tell if it's worth running. EF Tours built mart_metadata, a dbt-based observability solution tracking build consumption, Snowflake credit spend, Power BI usage, and downstream queries to quantify cost and adoption.
What you'll take away
- 01
EF Education First built a scoring system for every dbt model and test, based on execution time, credits consumed, and rows processed per second, normalized as a percentile rank against similarly sized peer models.
- 02
Sorting models into tiers, and surfacing low performers to their owning domain team, drove a 40% efficiency improvement on the heaviest pipelines and cut one of the biggest models from 10 minutes to 2.
- 03
A sharp distinction mattered most: usage and value aren't the same thing, a report checked a thousand times isn't necessarily valuable, and a good technical idea still doesn't adopt itself without real change management.
Session speaker
Adrien Ledoux
EF