Why context engineering belongs in the data warehouse

Teams are hitting a wall with their AI agents. Yours may well be one of them.

Contrary to most people’s assumption, the wall has little to do with their choice of frontier model. The problem is context.

AI agents need data, a lot of it, to return high-quality results. The fastest solution is to connect your agents directly to your data sources so they can take what they need. But many companies have seen this strategy break down.

This method doesn’t scale well, doesn’t deliver the results you want, and creates governance problems to boot. We’ll look at the benefits of storing your most important business context in your data warehouse, and how dbt makes that easy.

What context means to a model

Large language models (LLMs) like GPT, Claude, and Mistral are trained on large knowledge bases, such as the information available via the internet. This gives them a general knowledge base, along with language parsing and generation capabilities.

What it doesn’t give them is information about your business. This must be provided with every LLM request in what’s called the model’s context window. Well-selected context produces more accurate answers, reducing hallucinations and spurious assumptions.

There’s a flip side to this, though. First, context windows aren’t unlimited. While context window sizes are increasing, every model has an upper limit on how much data you can include in a single request.

Second, even if you can send more data, throwing a bunch of data at an LLM doesn’t necessarily equate to higher quality. Past a certain point, it can even lower quality.

More context also means more cost per call. Context gets broken down into tokens, such as words or word fragments. LLMs use tokens for pricing, since processing more tokens means using more compute. More tokens equal greater cost and longer query response times.

There’s no set size where request quality dips. There have been several studies, though, showing that such a point exists for most frontier models:

• In 2024, Databricks found that most models lost accuracy past a certain size, starting anywhere from 16K to 64K tokens depending on the model. The newest models it tested, such as GPT-4o and Claude 3.5 Sonnet, showed little to no decline.
• A 2025 study by Du et al. found performance dropped 13.9%-85% as input length grew, even when models could retrieve all relevant information.
• Tool bloat also hurts accuracy: one study found models picked the right tool 13.62% of the time with every tool loaded versus 43.13% of the time when only relevant tools were loaded.

Why having an MCP for every service breaks down

The default method for accessing context has become Model Context Protocol (MCP). It’s popular because it’s simple.

Using MCP, AI agents and source systems can negotiate capabilities (called primitives) at runtime. This obviates the need for a client to understand and create a static interface to a server’s API.

This is why the MCP docs refer to the protocol as “a USB-C port for AI applications.” With MCP, developers don’t need to spend days hard-coding access to the dozens (if not hundreds) of different data sources scattered across the enterprise.

The downside of MCP is that it breaks down in practice as the number of tools and data sources used increases. Anthropic, which created MCP as an open standard, documented this: in one blog post, it shows that a five-server MCP setup with 58 tools consumes about 55K tokens before the conversation even starts.

We’ve seen the same thing here at dbt Labs. We built a customer support chatbot using call transcripts obtained via the Gong API through MCP. Gong limits API calls to 3 calls per second and a total of 10,000 calls per day. We were repeatedly maxing that out, leading our agents to get throttled.

One reason we kept hitting these limits is that we had no reuse. Every time an agent needed a transcript, it asked for it even if it had already received it in a previous request.

Direct data access also raises potential data governance issues. A data source might have personally identifiable information (PII) that we don’t want to expose to agents, given their penchant for going rogue.

The alternative: engineer context in the data warehouse

We had to rethink our approach to storing business context. We experimented by doing what dbt does best: preprocessing the context and storing it in a data warehouse.

With this pattern, instead of accessing Gong API data ad hoc, we set up a context processing architecture that:

• Pulled in Gong data via Fivetran. Fivetran syncs only new and changed data and automatically pauses and resumes syncs when a source's API limit is reached.
• Modeled the context using dbt, leveraging its tried-and-true technology for using SQL statements to convert raw data into polished data models.
• Gave our agents access to the data warehouse.

This eliminated redundant and expensive calls to the call center transcript API. Instead of requesting raw data every time, our AI agents could now access transformed transcripts that included essential information.

Importing both structured and unstructured data

Most data teams are familiar with storing structured data, like metrics for their semantic layer, in their data warehouses. Using the warehouse to store unstructured data for AI agents remains a largely untapped technique.

The good news is that data warehouse providers have been working to make context engineering easy. Snowflake, Databricks, and other vendors offer built-in functions (like Snowflake Cortex’s AI_PARSE_DOCUMENT) that handle this work natively.

dbt further simplifies context engineering across data warehouses with dbt_context_engineering package. The package handles chunking, embedding, classification, extraction, and generation of text, turning it into useful context for a variety of use cases, such as sentiment analysis, product defect detection, and customer service chatbots.

What context engineering in the data warehouse gets you

Shifting context engineering to the data warehouse has numerous benefits over pulling data on the fly.

Lower token spend

For a while there, some companies encouraged employees to engage in “tokenmaxxing.” That lasted until the bill came due. Today, organizations want proof they’re getting a solid return on investment (ROI) for their LLM spend.

Preprocessing context makes it more compact. That means you’re sending less data to LLMs (using less of the context window) per call. This translates directly into lower LLM costs and thus higher ROI for your agentic solutions.

We saw these cost savings ourselves with our data warehouse-based context engineering solution. Token processing costs fell by over 50% with the new system.

Answers you can test

By storing context in your data warehouse with dbt, you also get all of dbt’s quality features. These include documentation, data tests, and automated data lineage.

This enables you to do something that’s difficult with raw context: test your agent’s results. Since your context layer exists in your data warehouse, you can test the output from agents against the facts stored there.

A security model built for reading

Source-system permissions are built around writing. MCP often connects with a user’s own, sometimes elevated, credentials. That potentially gives agents access to sensitive data, such as PII, that should be under lock and key.

Warehouse permissions, by contrast, are built around who needs to read the data. This gives data teams greater power to lock down sensitive information.

Context engineering in the data warehouse is good for your budget and good for governance.

Where MCP still fits

We’re not saying you should never use MCP. An MCP call is a live API call, and live API calls make sense when data is too volatile for long-term storage (for example, a stock price). MCP is also great when you’re initially experimenting with agents to discover which context is the most valuable for a given use case.

MCP is also extremely useful as a consumption model for business users. For example, once you have your context stored in a data warehouse, you can have an MCP server, like the dbt MCP server, that accesses the warehouse so stakeholders can retrieve information using natural language queries.

Modeling context for AI agents

The role of analytics engineering is evolving in the AI era. But the basics remain the same. Modeling context for AI agents follows the same process as preparing any other high-quality, governed dataset that you can create with dbt.

To get started, use metrics to identify your most expensive (and slowest) MCP sources. Model the data in dbt and bring it into a warehouse to realize immediate cost savings and speed increases.

If you’re not currently using dbt, you can get started today for free using the development platform of your choice.