Centrally defined metrics: The key to AI success

Everyone wants to leverage artificial intelligence to accelerate data-driven decision-making.

The blocker, as always, is the data itself.

While generative AI is new tech, it doesn't change the age-old computing principle of GIGO (Garbage In, Garbage Out). AI outputs are only as good as the data inputs. That's why, according to a Salesforce study on the state of data and analytics, 86% of business leaders believe high-quality data is essential for AI success.

Even before the advent of AI, however, your business has likely struggled with generating accurate metrics. Every team has its own approach to calculating fundamental metrics. That's confusing even for humans. When conflicting metrics are fed to AI systems, it can lead to poor decision making and ultimately stall AI adoption.

Before launching AI initiatives, it's imperative to have a single source of truth for your company's key metrics. In this article, we'll discuss how to build a centralized, governed semantic layer that feeds into your AI solutions and sets you up for success.

Why AI needs high-quality data and context

Artificial intelligence and machine learning use cases require high-quality, structured data, and governed context to function properly. Unlike traditional BI, where humans review dashboards and can spot obvious errors before making decisions, AI models (particularly AI agents) often act autonomously through automation. This makes subtle errors far more risky.

Our 2024 State of Analytics Engineering report found that the majority of analytics engineers are now managing data for AI model training. This shift reflects how quickly AI adoption has moved from experimental to production-critical across industries.

It's critical to get this data right. AI decisions now affect customer-facing products, automated systems, and business strategy. When an AI chatbot pulls from inconsistent revenue definitions across departments, it might tell Sales that Q3 was a banner quarter while telling Finance that performance was flat. These contradictions erode trust not just in the AI system, but in your data infrastructure as a whole.

Consider what happens when an AI-powered pricing algorithm uses outdated cost metrics, or when a customer service chatbot references product availability data that hasn't been refreshed in hours. The speed and scale at which AI operates means these errors can affect thousands of decisions before anyone notices.

These errors can also be hard (if not impossible) to track down once injected. That's because AI systems such as Large Language Models (LLMs) are complex neural networks that operate stochastically. It's not always clear how they arrive at their decisions—and developers can't step through their decision-making processes using normal application debugging techniques.

What makes this especially challenging is that AI doesn’t just need “clean data.” It needs context it can trust: what a metric means, how it’s calculated, what it can be sliced by, what it can be joined to, and whether it’s approved for use. Without governed context, AI systems will fill in the gaps, often incorrectly.

The root problem: metrics chaos

Modern data infrastructure has become increasingly complex. According to Forrester research, 61% of organizations use four or more business intelligence tools, and 25% use ten or more. Each tool becomes a silo with its own metric definitions, creating what we call "metrics chaos."

Your revenue metric is calculated one way in Tableau, another way in Looker, and yet another way in your custom applications. Each definition made sense at the time it was created, but as they've diverged, no one can definitively say which is correct. Finance uses one number, Sales uses another, and your executive dashboard shows a third.

This chaos has several root causes that compound over time.

Version drift

Metrics evolve as your business changes. You might update how you calculate customer lifetime value to exclude certain transaction types, or modify your churn definition to better reflect your new subscription model.

The problem is that old definitions persist in some tools while new ones appear in others. There's no single source tracking which definition is current, which is deprecated, and which teams are using which version. Your marketing team might be optimizing campaigns based on metrics that your data team abandoned months ago.

Duplication and inconsistency

Teams repeatedly rebuild the same metrics across different tools and platforms. Each time they rebuild, subtle differences in calculation logic creep in. Maybe one version rounds differently, or includes a slightly different set of transaction types, or uses a different lookback window.

The names might differ, too. What Finance calls "gross revenue" might be what Sales calls "total bookings" and what the executive team knows as "sales volume." Or the names might be obscure and technical—like metric_rev_001—rather than defined in clear business language. When stakeholders see conflicting KPIs in different dashboards, trust in data erodes quickly.

The maintenance nightmare

When metrics definitions are scattered across multiple systems, updating them becomes a monumental task. Changing a single metric might require touching five different tools, each with its own syntax, testing environment, and deployment process.

Testing changes becomes impractical at scale. You'd need to verify that the updated metric works correctly in every system that references it, and that downstream reports and dashboards still function as expected. Teams start avoiding making necessary updates simply because the complexity isn't worth the effort. Your metrics calcify, becoming increasingly disconnected from business reality.

Why traditional BI doesn't work for AI

Traditional business intelligence tools were designed with a human-in-the-loop approach. Someone reviews a dashboard, considers the numbers in context, spots anomalies, and decides whether to take action. Humans serve as the final filter for data quality.

AI agents eliminate that filter. They make decisions and take actions without human review, at a speed that makes manual oversight only possible after the fact, via auditing and logging.

An AI-powered pricing algorithm might adjust thousands of products before anyone realizes it's using outdated cost data. A marketing AI might send millions of personalized messages based on incorrect segmentation rules. This speed of decision-making means errors propagate instantly across your business.

Traditional BI tools also weren't designed to provide the explicit, machine-readable metric definitions that AI tools need. A dashboard might be sufficient for a human analyst. By contrast, an AI agent needs structured metadata about what each metric means, how it's calculated, what it can be sliced by, what it can be joined to, and when it's appropriate to use.

The Analytics Development Lifecycle (ADLC) emphasizes treating data as software—with versioning, testing, and deployment processes. AI implementation exposes technical debt in your data infrastructure that was previously invisible. When humans were the consumers, they could work around data quality issues. AI systems can't.

Solution: Centralized, governed metrics via a semantic layer

A semantic layer provides a unified, business-friendly representation of your data. Think of it as a translation layer that sits between your raw data and your analytics tools, defining metrics once in clear business terms.

But for AI, the goal isn’t “just a semantic layer.” What AI actually needs is governed semantic models: machine-readable business definitions that are version-controlled, tested, documented, reviewed through CI/CD, access-controlled, and auditable, so AI systems can rely on approved logic instead of guessing.

The architecture follows a hub-and-spoke model: you define metrics once in the semantic layer, and they're consumed everywhere—from traditional BI tools to embedded applications to AI agents. Metrics are defined in code alongside your data transformations, making them version-controlled and testable using the same principles you apply to your data pipelines.

The semantic layer exposes these metrics via APIs and the Model Context Protocol (MCP) server to all downstream consumers. For example, you might define your revenue metric once. That same definition then powers your Tableau dashboards, your embedded analytics application, and your AI chatbot. When you need to update the definition, you change it in one place, and the change propagates everywhere. This means these governed definitions stay interoperable as your BI tools, applications, and AI systems evolve, so you don’t have to re-implement metric logic every time your stack changes.

This centralization doesn't just reduce duplication—when it’s combined with governance, it makes metrics governance practical. With governed semantic models, definitions live in one place where they can be version-controlled, tested, documented, reviewed and deployed through CI/CD, access-controlled, and auditable. You can track which definition is current, who approved it, when it was last updated, and which systems are consuming it.

The benefits of centrally defined, governed metrics for AI

Centralized metrics unlock several critical capabilities for AI systems that directly impact business value and operational efficiency.

Consistency means AI agents always access current, approved metric definitions. Your AI doesn't need to guess which revenue calculation is correct or reconcile conflicting definitions. It uses the canonical version that your data team has validated and governance team has approved.

Governance becomes granular and practical. You can implement role-based access controls at the metric level, ensuring that sensitive financial metrics are only available to authorized AI systems. You can require approval workflows for metric changes, and audit who accessed which metrics and when.

Auditability provides the transparency you need for regulated industries and high-stakes decisions. You can track exactly which performance metrics your AI systems are using and when they access them. If an AI makes a questionable decision, you can trace it back to the specific metric definitions and data it relied on.

Velocity improves dramatically. You can ship new metrics to AI applications without rebuilding integrations. Your AI chatbot automatically gains access to new product metrics the moment you add them to your semantic layer. There's no separate deployment process for each consuming system.

Interoperability keeps you flexible. You can adopt new tools—new BI platforms, new agent runtimes, new applications—without re-implementing metric definitions each time. Your governed semantic models remain the stable source of truth.

Driving centrally defined, governed metrics with dbt

dbt acts as a data control plane, driving data quality and trust regardless of where your data lives. With dbt, you define semantic models and metrics on top of those governed dbt assets in the modeling layer (your dbt project). Crucially, in dbt these semantic definitions are in code. Semantic models and metrics are configured in YAML files within your dbt project repo.

The workflow follows analytics engineering best practices your team already uses for transformations. You define transformations in SQL, test them quickly on dev machines with dbt Fusion—which delivers up to 30x faster parsing—then add testing and documentation before deploying changes. You manage deployment through CI/CD pipelines, ensuring that every metric change is reviewed, tested, and versioned. That’s what turns “a semantic layer” into governed context that can safely power both BI and AI

The dbt Semantic Layer provides purpose-built tooling for defining a single source of truth for organizational metrics, grounded in semantic models. You specify how to calculate each metric, which dimensions it can be sliced by, and what aggregations are valid. The semantic layer then translates metric requests into the appropriate, optimized SQL for your warehouse, including the joins and aggregations needed to answer a question correctly. Key features include:

• Metric definitions and semantic models as code (YAML): Define calculations and manage business logic once using familiar SQL syntax with reviews and change history.
• Multi-dimensional modeling: Specify which dimensions, entities, relationships, and filters apply to each metric so AI and BI isn’t guessing.
• Query optimization: The semantic layer generates efficient, correct SQL regardless of how metrics are consumed
• Access controls: Govern which teams can access which metrics
• Version management: Track changes to metric definitions over time through Git and CI/CD.

You can expose dbt semantic layer governed metrics and context to AI systems using the dbt MCP Server. This allows language models to query your metrics directly, with the dbt semantic layer ensuring AI systems always use the correct, approved definitions.

Best practices for getting started with a semantic layer

You don't need to centralize every metric on day one. Start with the critical few that drive key decisions and business objectives.

Identify 5-10 metrics that matter most to your business. These might include revenue, customer acquisition cost, churn rate, and other KPIs that executives review regularly. Focus on metrics where inconsistency currently causes real problems—perhaps, for example, where Finance and Sales regularly disagree on numbers.

Apply ADLC principles to metric development. Plan which metrics you'll centralize first and why. Develop the metric definitions in code, with clear documentation about calculation logic, and test them against known data to verify accuracy. Then, use a CI/CD deployment process to safely deploy changes to production and monitor their behavior.

Integrate with AI workflows from the start. Expose metrics through APIs that your LLMs can consume. Use connectors such as the dbt Model Context Protocol (MCP) server to connect MCP-aware clients to your semantic layer, ensuring that your AI applications always have access to your single source of truth and governed context.

Define a governance framework that grows with your semantic layer. Establish clear ownership for each metric and create an approval process for metric changes—perhaps requiring sign-off from both the data team and the relevant business stakeholder. Set up production testing and data health signals so you can issue alerts on metric quality issues, like sudden unexpected changes in values or failed data tests.

From firefighting to proactive AI enablement

Organizations that want to succeed with AI must first get their metrics under control.

A governed semantic layer isn't a nice-to-have. It's foundational infrastructure. Without it, you'll spend your time firefighting data quality issues and explaining why different systems show different numbers, rather than delivering AI-powered insights that drive business value.

The good news is that you don't need perfect data to start. Begin with your most critical metrics, apply consistent governance, and expand coverage over time. AI adoption won't wait for you to clean up every data silo and reconcile every definition. But it will expose your data quality issues faster and more publicly than ever before.

Start small, but start now. The organizations that thrive in the AI era will be those that build on a foundation of trustworthy, centrally defined, governed metrics.

To learn more about how to transform your metrics chaos into AI-ready infrastructure, contact us for more info on how dbt can help you build the governed semantic layer your AI systems need.