Your dbt tests passed. Why did the metric still move? Making semantic stability a platform guarantee

Every test was green — freshness, schema, not_null, accepted_values — and the retention number was still wrong, three separate ways.

None of them were bugs. Each was a decision that looked correct in review: what "retained" means, when a cohort is old enough to score, and which records belong in the cohort at all. Tests validate the structure of an answer. They say nothing about whether it answers the right question — and an agent writing your SQL can't tell the difference either.

Three decisions, what each did to the number, what shipped to fix them, and the gap that's left: those decisions now live somewhere reviewable, but nothing fails the build when meaning changes.

What attendees will learn:

• Why a metric can be confidently wrong while every test passes
• The three semantic decisions in any cohort metric: definition, denominator, membership
• Fixing them in dbt: lifecycle-based definitions, right-censoring flags, identity joins with deterministic dedup
• Why a date-proximity join is a hypothesis, and the one check that exposes it
• What has to be true before you let a generated model define a metric