Beyond the basics: Running dbt at scale on Microsoft Fabric

This session walks through building a production-grade medallion architecture (bronze → silver → gold) on Microsoft Fabric, orchestrated end-to-end through a dbt job. We'll cover how to structure dbt models for each layer — raw ingestion and cleansing in bronze, conformed and deduplicated data in silver, and governed, BI-ready tables in gold — and how to scale that structure as data volume, model count, and team size grow.

You'll see how to design dbt jobs that sequence bronze, silver, and gold builds with correct dependency ordering, enforce testing and freshness checks at each stage, and scale cleanly from a handful of models to hundreds without becoming unmanageable. We'll also cover incremental models and snapshots for handling growing data volumes efficiently, contract testing at layer boundaries to catch quality issues early, and CI/CD practices for keeping the whole pipeline reliable as it scales.

Beyond the dbt job itself, we'll cover how to operationalize the entire pattern using Microsoft Fabric Data Pipelines — triggering and monitoring dbt runs, handling retries and alerting, and integrating dbt execution into broader orchestration alongside ingestion and downstream refresh tasks. We'll show how to make this operationalization metadata-driven, using a config-based approach (e.g., a control table or metadata file describing sources, layers, dependencies, and schedules) so that new models, sources, or layers can be onboarded by updating metadata rather than hand-authoring new pipeline logic — keeping the orchestration layer scalable and maintainable as the project grows.

You'll leave with:

• A reference medallion architecture pattern built for scale on Microsoft Fabric
• A dbt job design that orchestrates bronze → silver → gold with clear dependencies
• Incremental modeling and testing strategies that hold up as data and model count grow
• CI/CD practices for scaling a multi-layer dbt pipeline with confidence
• A pattern for operationalizing dbt with Fabric Pipelines, driven by metadata for scalable, low-maintenance orchestration