From analytics engineer to context engineer

Most companies are taking shortcuts in their enterprise AI rollouts today, and falling into a classic trap their data teams are already deeply familiar with. Taking the “easy route”—wiring AI straight to MCPs provided by vendors, connecting them directly to source data providers—is collectively costing companies billions in token consumption, vendor API charges, and lack of data ownership and portability.

At dbt Labs, we initially were no different. We used vendor MCPs until we discovered how significant these hidden costs actually were.

That's when we realized that the most efficient, scalable way to connect AI to data was what we’ve evangelized all along: to centralize our data and engineer context directly within our data warehouse, giving us a massive opportunity to own our AI’s context layer and expand our data team’s remit.

How it started

As Claude rolled out in our organization, one of the top user groups was sales connecting to Gong, Salesforce and other rich context sources to analyze their deals. When our GTM teams requested customer insights to prevent churn and identify high-value accounts, of course we agreed and allowed them to analyze Gong data by connecting AI agents directly via the native MCP.

With hundreds of salespeople using this data many times a day, costs added up quickly. Analyzing a single call consumed up to 50,000 tokens; that's $0.25 in token cost just for AI to read a single transcript. We had 500M+ tokens worth of Gong transcript data that we could tap into. With many salespeople running many Claude sessions throughout the day, our total AI spend went up significantly at scale.

We knew we needed a new approach. That’s when we decided to ingest the raw Gong data into our data warehouse and used data modeling to summarize each call. By actively context engineering the transcript data to eliminate noise and extract signal, we shrank data volume by 20x and slashed token consumption on a 60-minute call from tens of thousands to just a few hundred token. The question was, if we can transform any of our data into more meaningful context, how could we engineer the smallest, most meaningful context layer that still answers most queries?

How dbt reduced token costs

When our data team had previously modeled Gong data for BI, the ROI wasn't there. Now, though, Claude and ChatGPT agents unlocked new use cases that let us do qualitative data analysis at scale, and the call transcript table became one of the highest value assets in the data warehouse.

Sourcing context through the data warehouse is vastly more efficient than pulling it straight from MCPs. Serving that call transcripts from warehouse summaries dropped token costs by roughly 98% while maintaining or improving context quality.

In this blog series, we will share core patterns for efficiently modeling trusted context for AI, an end-to-end technical walkthrough of our Gong pilot, the fundamentals of reading data once to serve every agent, and why you can transform traditional analytics engineering into context engineering without changing your stack.

Qualitative data analysis at scale with engineered context

Before AI, working with text data at meaningful scale required either simple and ineffective regex techniques like keyword matching or advanced statistics and data science. Maintaining complex transformations for qualitative data sources like call recordings or support tickets was just not feasible for data teams.

As our Gong breakthrough shows, though, we have crossed a threshold where previously ignored data and data sources can now form some of the most useful and valuable agent context. Teams can build models and highly effective context layers for AI, applying the general principles we pioneered with analytics engineering as long as the techniques are applied with AI in mind.

Analytics teams have always modeled quantitative data for BI. Context engineering is just modeling data for agents, and that data is far more than metrics:

A dashboard needs metrics. Agents need context.

Untapped qualitative data is where AI truly pays off. Instead of just serving structured data to dashboards, data teams can now use unstructured data like PDFs and JIRA tickets to engineer context. LLM advances make it possible to move, store, model, and process these files within the data warehouse and use them as agentic AI context.

This is a new way of thinking for a lot of analysts because, historically, data organizations have been allergic to unstructured data like transcripts, and for good reason: our technology was not built to support it.

We were forced to change because we were running out of Gong API calls. Everyone was asking very similar questions on very similar data, going right to Gong saying give me all of my transcripts, now summarize each transcript. The direct MCPs circumvented our entire traditional data modeling and warehousing world to repeatedly query raw data, which created redundant token costs and caused API constraints. This was when we realized hey, you know, we actually already have this process whereby we model data into trusted, governed, more useful forms. Why don’t we apply that for data for AI?

Following our merger with Fivetran, we realized that moving and modeling unstructured data is already in our wheelhouse. Platforms like Salesforce, Zendesk, or Gong now provide critical business context. Modeling this qualitative data and caching pre-built AI summaries in the warehouse delivers reliable, deterministic context while eliminating multiple token-heavy MCP calls, tool proliferation, and API constraints.

Users seamlessly access this modeled data through a single context connector in the context layer via MCP. This drastically lowers token costs, simplifies the user experience to a simple chat, and opens up reusable, community-driven data patterns across the entire organization.

How to do cost-effective context engineering

There’s no magic in this approach. Data teams turn raw data, whether quantitative, qualitative, or semi-structured, into agent context through the same processes they already practice without materially changing the stack:

• Compress: reduce a large corpus to the smallest forms that contains only what’s relevant
• Enrich: join data to other relevant information so that it’s easier to access everything needed, like opportunity details and qualitative deal history.
• Describe: provide information around what the data means and when it’s applicable
• Govern: decide exactly what each agent is allowed to see to do its work

The data team models the context once, and every tool and agent reads from the same trusted context layer. The analytics engineer is now the context engineer. It’s the same skillset, except beyond modeling metrics you’re also mapping the entire data estate and every data source is now in play.

All of this happens without materially changing your data stack. The data team builds a structured shared context layer for the entire organization, functionally layering AI over your original data stack in the same way that dbt is currently layered over your data warehouse. The whole company is now the consumer because every team and the agents they use can access this context layer, not just people who write SQL queries.

Functionally, though, how do you move, transform, and manage structured, semistructured, and unstructured data into the context layer?

1. Getting data, both traditional tables and unstructured files, into the data warehouse (Snowflake, BigQuery, Databricks) is Fivetran's job. If tabular data needs further preparation (filtering, denormalizing, joining, and aggregating), then dbt allows the creation and execution of that transform logic in an open and portable way.
2. Automating pipelines that utilize the SQL-based AI functions of your destination is dbt’s job. For example, Snowflake Cortex provides SQL functions such as AI_EMBED, AI_COMPLETE and AI_PARSE_DOCUMENT that can be executed as part of, and orchestrated by, dbt models. BigQuery and Databricks offer similar functionality.

Raw text fields, structured data from files like spreadsheets, and even replicated unstructured files like PDFs become queryable, retrievable context available to both human data users and AI agents in the shared context layer. The documents and data sources they seek information from are already in the same platform, moved and indexed automatically by Fivetran and processed as part of your dbt models.

The stack beneath the context layer remains the same: Fivetran moves your data; dbt models it. Now, though, this includes the unstructured context that makes AI systems powerful. All the Salesforce email bodies, Jira ticket attachments, call transcripts, and contract PDFs that have always existed but never made it into a pipeline are now fully referenceable, trusted context.

Best practices are beginning to emerge

We are continuing to build the vocabulary, packages and open source projects that analytics engineers will need to use to do context engineering. We’ve come up with a few best practices we can share, with much more to come. Here’s a preview of some our team uses:

• Don’t aggregate context, generate it: Use warehouse-native AI to create new context by parsing, chunking, embedding, transforming and joining related information into modeled data objects that agents can query. Aggregating can cause context to get lost, so maintaining the context quality through the whole pipeline is critical.
• Context layer architecture: One shared, structured and governed, open-format canonical context layer that every engine can write to and every agent can read from. The same single source of truth the data industry has always focused on is now pointed at AI as the primary consumer.
• Read once, write many: Reads are the most expensive part of context engineering. Do the expensive read with an LLM once, in batch processing where it’s cheaper; then serve the modeled form to every agent in the context layer
• Incremental context maintenance: Agents do actions, and they need correct, current information to act on. Context can’t simply be snapshotted, it must be updated as new events and information come in. Incremental models that update context’s current state (with pipelines built to capture the state changes so that they are auditable) are critical.

Until recently, the context path of least resistance was to just take the raw API and connect the MCP server. Now, data analysts can step in and say, "Use our existing data. We'll augment that with some of the AI capabilities you're asking for, process it once, and make it accessible to everybody to use an unlimited number of times.” Now, the data team owns AI and becomes the mission-critical team for the next era of businesses.

Fivetran + dbt Labs are building the data foundation for agents you trust. Join us at dbt Summit, where data practitioners and leaders come together to shape the future of data and AI.