How Figma turned dbt tests into consistency checkers that catch 100+ bugs a month

Madison Kohls, a data scientist supporting Figma's growth and monetization teams, walked through how her team built what they call consistency checkers: dbt singular tests wired into alerting, task tracking, and investigation tooling to catch data and product bugs before they cause real damage.

The story starts with Figma's pre-IPO billing remodel. Figma was moving off a "true-up" model, where customers could add users to unpaid seats and settle up later, to a request-approval model where every seat is approved up front.

With more than 500,000 paid plans and 5 million paid users on the line, and the accuracy demands of an upcoming public offering, Figma's engineering teams needed a way to catch billing mistakes that ordinary unit tests and bug bashes wouldn't surface.

A consistency checker is just a dbt singular test, similar to the pattern in dbt's own docs for adding data tests to your DAG, but wrapped in infrastructure that makes the results actionable. Dagster runs the checks on a schedule against Snowflake, and failures get stored individually and in aggregate.

A second layer decides what to do about it: PagerDuty and Slack for on-call alerting, Asana and Linear for task tracking, and Hex for investigating what actually broke, including a write-back path for engineers to mark a failure as expected.

During the remodel, Figma's teams built more than 100 consistency checkers and caught five or more billing discrepancies serious enough to have blocked either the remodel or the IPO. Today, Figma runs more than 450 consistency checkers across 15 distinct product and business functions, catching over 100 real code and data issues in a typical month.

The same pattern now covers Figma's AI credit system, more than 45 distinct AI features used by over 700,000 customers.

The checkers weren't precise out of the gate. Kohls said 67 percent of early failures were false positives, split roughly between stale data syncs, wrong SQL logic, and permission gaps that scrubbed some records but not others.

Double-checking failures against a second, later sync, using AI agents to help write and iterate on the SQL, and getting legal sign-off to use fuller datasets brought the false positive rate down to 33 percent.

In the Q&A, Kohls said the team draws a line between routine dbt tests, which stay in the DAG to catch model-breaking issues, and consistency checkers, which are reserved for tests that surface product or business-facing problems. Ownership sits with software engineering the way unit tests do, with data science stepping in as a partner on more complex SQL.

Watch the full session, plus every keynote and breakout from dbt Summit 2026, on demand. Watch dbt Summit 2026 on demand →