Why a semantic layer is the difference between a 96% and an 80% accurate AI analyst

Alex Noonan, senior developer experience advocate at dbt Labs, used his dbt Summit 2026 session to trace the long, repeated failure of self-service analytics, from a 1971 natural-language moon-rock query system through 1990s OLAP cubes and a string of more recent natural-language BI features that never quite stuck. His argument for why this attempt is different: large language models have finally solved the "teaching the machine your words" half of the problem.

What's left is grounding, meaning teaching the machine what your words actually mean inside your business. He backed that up with a benchmark from dbt Labs' own developer experience team: a model asked to answer questions through a semantic layer scored 96 to 100% accuracy, compared to 80 to 91% when the same model wrote raw SQL directly against a database schema.

Noonan’s framing of that gap was direct: every point of difference is a confidently wrong number landing in someone's Slack channel. He pointed to three converging approaches teams are using to ground an AI analyst: a semantic layer with pre-defined metrics, an ontology-style entity and relationship model in the style of Palantir, and an agent context layer that assembles relevant context at query time, which he described as the direction Fivetran and dbt are building toward together.

Most of the talk focused on what Noonan called the unglamorous fundamentals that separate a demo from a durable system: data quality and observability practices like tests, freshness checks, and anomaly detection; documentation written to be read by machines as much as people; and consistent data modeling.

She also draws a split within context engineering between "just in time" context, meaning stable metadata like a semantic layer or agent skills that doesn't change much, and "just in case" context, meaning the search infrastructure needed for high-volume unstructured sources like call transcripts and support tickets.

On trust specifically, Noonan recommended keeping the chat interface in a public channel like Slack so mistakes and corrections happen visibly rather than in a private thread, and cautioned against relying too heavily on "LLM as judge" evaluation, since a judge model shares the same blind spots as the model it's grading.

She cited a figure that roughly half of generative AI pilots get abandoned after proof of concept, and pointed to this year's State of Analytics Engineering report, in which about half of dbt users said they're actively building toward conversational analytics. Two lines Noonan returned to: "the worst thing that you can do is give somebody a confidently wrong answer," and "silent failures are the killer in any IT system."

Noonan closed with a live demo of a personal AI trading assistant he built on dbt's MCP server, pulling SEC filings and technical indicators, noting that a trade it had recommended had closed profitable the day before his talk.

Watch every keynote and breakout from dbt Summit 2026 on demand. Watch dbt Summit 2026 on demand →