The semantic debt crisis no one is talking about

This is a guest post from Dustin Dorsey, Senior Director of Data Engineering at phData and co-author of Unlocking dbt. He works with enterprise teams to build the data foundations that make AI reliable at scale.

Picture the scene. Two teams are presenting to the same leadership group. Same company. Same time period. Same metric. Different numbers.

Both teams defend their number with confidence. Both can show their work. Both are, by their own definition of the metric, correct.

The meeting derails. Someone from finance says the sales team's number is wrong. The sales team says finance is using the wrong date field. Someone points out that marketing has a third version that does not match either of them. An hour passes. Nothing gets decided. The executive sponsor asks someone to "get aligned on the numbers" before the next meeting, and everyone leaves knowing that conversation will take months if it happens at all.

This is not a data quality problem. The data is accurate. The pipelines ran correctly. The numbers are precisely what they claim to be.

It is a design problem. Business meaning was never encoded into the data itself, so every team built their own version of the truth, and now there are three of them. Each defensible, none authoritative, all quietly costing the organization more than anyone has ever calculated.

Dustin goes deeper on the data design decisions that make AI reliable in production: Your AI isn't broken. Your data model is.

This is what semantic debt looks like

Technical debt is a concept most engineers know well. You take a shortcut to ship faster, and you pay for it later in maintenance, refactoring, and fragility. Semantic debt works the same way, except the thing being deferred is not code quality. It is shared meaning.

Every time a team builds a metric without encoding a canonical definition, that is a withdrawal from the semantic account. Every time a dashboard hardcodes a filter that belongs in the data layer, that is a withdrawal. Every time a business concept gets defined slightly differently in two separate pipelines, both of which work fine in isolation, that is a withdrawal. No individual shortcut looks catastrophic. The accumulation is.

In most organizations, semantic debt has been building for years, sometimes decades. It was built quietly, because humans are remarkably good at compensating for it. A finance analyst knows their "revenue" is not the same as the sales team's "revenue" and calibrates accordingly. A data engineer knows which version of a customer record is authoritative for which use case. A business intelligence developer knows to add that one filter that nobody can explain but everyone agrees must be there. These workarounds are so ingrained they stop feeling like workarounds. They feel like just how things work.

What they actually are is institutional knowledge plugging structural gaps that should have been designed out years ago.

The meeting AI cannot call

In a human-driven analytics environment, semantic debt gets resolved through a specific, recognizable process. Two teams disagree on a number. Someone calls a meeting. In the meeting, each team explains their logic. The group agrees (usually informally) on which definition is correct for which purpose. Someone writes a note in a Confluence page that nobody will find in two years. The organization moves on, with a slightly better shared understanding that lives, once again, in people rather than systems.

This process is inefficient. It is slow. It only surfaces the conflicts visible enough to generate a meeting. But it more or less works, because the scale of human-driven analytics is limited enough that the gaps can be managed.

AI cannot call that meeting.

When an AI system is asked a question that depends on a metric your organization has defined inconsistently, it does not pause, identify the conflict, schedule a working session, and wait for consensus. It picks an interpretation and produces an answer. Confidently. Without flagging which version of the definition it used. Without knowing that a different version exists. Without any signal to the downstream consumer that the answer may depend on a definitional assumption that three teams would argue about if they knew it was being made.

The conflict that your organization has been managing through meetings and tribal knowledge for years does not disappear when AI arrives. It gets made at machine speed, at scale, without anyone in the room to catch it.

Where meaning lives in most organizations

If you were to trace the authoritative definition of your organization's most important metrics, you would find them in some combination of the following places: a Confluence page last updated eighteen months ago, a comment in a SQL file that three engineers have each interpreted slightly differently, the institutional memory of a senior analyst who has been with the company for eight years, a Slack thread from 2022 that was pinned for a while and then wasn't, and a spreadsheet that lives on someone's desktop that nobody else knows about.

This is not an exaggeration, and it is not a sign of dysfunction. It is the default state of organizations that built their data stacks to move fast and deliver value quickly within specific domains. Meaning accumulated in people because people were always there to apply it. Encoding it into the data structure itself was extra work that did not obviously improve the metric that mattered, which was whether the dashboard loaded.

The problem with meaning living in people is not that people are unreliable. It is that meaning needs to travel to places people cannot go.

It needs to travel to the AI system that a business user is querying at 11pm trying to understand why their numbers look different from last week. It needs to travel to the automated pipeline that is making decisions based on customer behavior without a human in the loop. It needs to travel to the new analyst who joined the company six months ago and has never heard the story of why that filter is always set to exclude refunds. It needs to travel to every downstream consumer of the data, reliably and consistently, every single time.

People-stored meaning cannot do that. Only data-encoded meaning can.

The stakes are higher than they appear

Most organizations experience semantic debt as an inconvenience. Meetings that take longer than they should. Reports that require manual reconciliation before they can be shared. Analysts who spend a third of their time validating numbers before they trust them enough to use. These costs are real, but they are diffuse and largely invisible on any financial statement.

AI changes the cost structure entirely.

When AI is operating on data where meaning is inconsistently defined, the outputs are not just inconvenient, they are unreliable at a scale no human team could produce. An analyst who does not know which revenue definition to use asks a clarifying question. An AI that does not know which revenue definition to use picks one and answers three hundred queries with it before anyone notices the problem.

The other shift is who is affected. Traditional analytics tools were used primarily by trained practitioners who understood the limitations of the data they worked with and knew, from experience, where the landmines were. AI-powered interfaces lower the barrier dramatically. Business users who have never seen the underlying data model, executives who expect answers to just be right, external stakeholders in some cases, all of them are now interacting with data whose meaning was never designed to be self-evident. Semantic debt that was manageable when experts were the only users becomes genuinely dangerous when anyone with a natural language prompt can query your data directly.

You have a preview, not a problem

Here is the argument I want to make directly: if your most important business metrics mean different things to different teams today, you do not have an AI problem yet. You have a preview of one.

The meeting where two teams present different numbers? That is semantic debt becoming visible. It is uncomfortable and inefficient, but it is still being caught. Someone is in the room. The conflict surfaces. It gets resolved, however imperfectly.

The question is whether your organization addresses the underlying cause intentionally (before AI amplifies it) or reactively, after AI has already made it impossible to ignore.

Reactive is expensive. Not just in technical remediation time, but in organizational trust. The erosion of confidence in AI outputs is one of the hardest things to reverse once it begins. Business users who receive inconsistent answers stop trusting the system. Executives who got burned by a bad AI-generated number become skeptical of every AI-generated number. The technology gets blamed for a problem that the technology did not create, and the actual cause (the accumulated semantic debt in the data layer) stays unaddressed because the room full of people who saw the symptom did not get far enough upstream to see the disease.

The organizations that act now have a significant advantage. Not because they will finish building perfect semantic foundations before anyone else because that is not how this works, and the organizations claiming they will are deceiving themselves. The advantage is that they will start building them as a deliberate strategic investment rather than as an emergency response to a crisis that has already damaged trust.

Making meaning a system property, not a people property

Think about what it would actually take for a new analyst to answer the question "what is our monthly active user count" correctly in their first week. They would need to know which table is authoritative, which definition of "active" the business uses, which date field governs the period, which user types to exclude, and probably the history of why each of those decisions was made. None of that is in the data. It lives in people. The new analyst calls a meeting. Someone explains it. They write it down in a place only they will ever look. Two years later, a different analyst asks the same question and starts over.

This is the loop that data-encoded meaning breaks. When business logic is enforced in the structure itself (in how grain is declared, in how relationships are modeled, in what the transformation layer enforces) a new analyst and an AI system get the same correct answer without needing to know the history. Not because someone documented it well. Because the structure made the wrong answer impossible to produce.

The distinction matters because meaning needs to travel. It needs to travel to the new analyst who joined six months ago, to the automated pipeline running without human supervision, to the AI system answering questions at 11pm. None of those consumers can call a meeting when they encounter ambiguity. They either get a consistent answer because the structure provides one, or they get a defensible guess because it does not.

This is also not a one-time project. It is ongoing stewardship. Meaning drifts as the business changes. New products launch. Definitions evolve. Mergers happen. The organizations that stay ahead of semantic debt are not the ones that did a modeling exercise in 2022 and called it done. They are the ones that treat their semantic foundation the same way they treat their data pipeline. As infrastructure that requires ownership, investment, and deliberate evolution.

What most organizations are missing is not the technical ability to do this. It is the organizational will to treat it as infrastructure rather than overhead. That shift (from viewing semantic work as a documentation task to viewing it as a platform investment) is what separates the organizations that will navigate the AI era well from the ones that will spend the next several years explaining why the answers keep changing.

What you should be asking right now

The diagnostic for semantic debt is not a technical audit. It is a business conversation.

Pick your organization's three most important metrics. Ask five people from different teams to define them. Not to query them, but to define them. To explain what counts, what does not, what edge cases apply, and how they should be calculated. If you get the same answer from all five people, you are in better shape than most. If you get five different answers, each defensible, you are looking at your AI risk surface.

The follow-up question is where the decision lives: do you want to address this before the AI initiatives your organization is investing in expose it at scale, or after?

There is no neutral choice here. Semantic debt compounds. Every month it goes unaddressed is another month of diverging definitions, additional teams building their own versions of the truth, and growing distance between where your data is and where it needs to be for AI to operate reliably on top of it.

Addressing semantic debt is not just about fixing individual models, though. It requires rethinking the fundamental purpose of the data platform itself. Most organizations are building platforms designed to store. What the AI era requires is platforms designed to reason. That is the argument in the next post and it is a bigger shift than it sounds.

If the patterns in this post feel familiar, Building the Foundational Layer for Reliable AI on Structured Data goes deeper on the structural conditions that separate data environments where AI operates reliably from the ones where it keeps surfacing the same unresolved ambiguity. Worth reading before your next AI planning conversation.

Where dbt and phData fit

dbt is one of the few tools in the modern data stack where semantic debt can be paid down incrementally and sustainably. Its combination of model-layer definitions, the dbt Semantic Layer for centralized metric governance, and documentation gives teams a practical path to encoding meaning as part of how transformations are written and maintained. phData helps teams put that into motion by working through the harder operational pieces alongside dbt: clarifying definitions, reshaping models, and building the delivery discipline needed to make semantic consistency hold up beyond a single project or team.