Conversational analytics: building AI-powered analytics people can actually trust
Overview
This session explores production conversational analytics: semantic layers and vector indexes, using dbt to manage semantics and vectors in Snowflake and BigQuery, and foundational requirements like clean models, documentation, and observability. Learn about trust beyond model accuracy and the gap between demos and production.
What you'll take away
- 01
A model answering questions through a semantic layer scored 96-100% accuracy, compared to 80-91% when the same model wrote raw SQL directly against a database schema.
- 02
Grounding, not language, is the unsolved half of self-service analytics: LLMs already understand your words, the gap is teaching them what those words mean inside your business.
- 03
Two recurring lines: "the worst thing you can do is give somebody a confidently wrong answer," and "silent failures are the killer in any IT system" — arguing for unglamorous fundamentals like tests and documentation over flashier fixes.
Session speaker
Alex Noonan
dbt Labs