A guide to implementing AI data pipelines

There’s a startling pair of statistics in dbt’s newly released 2026 State of Analytics Engineering report: 72% of teams prioritize AI-assisted agentic coding for data work, while only 24% prioritize AI-assisted data pipeline management.

Here are my thoughts on why this is happening and, if this is the case in your own org, how to go about building AI into your everyday workflows.

Why AI pipeline management is lagging behind AI coding

Looking at these numbers in relation to each other really shows the difference between AI adoption in terms of giving developers AI tools to work with, and AI integration by working AI into the data infrastructure itself.

AI-assisted coding is very self-driven. You can be on your individual system, working on your own, using an AI coding tool on whatever dbt project you personally have pulled down. But using AI for pipeline management, like looking for job errors and feeding that data into an agent, has to happen at the team level. I can absolutely spin up an agent in a five-minute Claude Code session, but actually building the bots that would monitor a data pipeline takes time, dedication, and collaboration from basically the whole team.

One of the largest clients I'm currently working with has a team of 80 engineers throughout their entire company that touch their dbt project. But when it comes to actually orchestrating that project, it's a team of maybe five engineers managing the orchestration out of the dbt platform. I think that's why we're seeing a slower uptake on using AI to assist data pipeline management, because it has to go through different approvals and the team has to be aligned on what exactly they want that to look like.

Ultimately, the difference in those numbers are really about accessibility: AI coding is individual, personal, and easy for developers to adopt, versus actually building and managing AI pipelines in production as a team.

Your data stack is already AI-ready

The typical data architecture for many orgs today follows the ELT pattern: data is extracted from source systems, loaded into a data warehouse, and transformed for downstream consumption by business users.

A lot of bringing AI into your daily workflows does not necessarily change that stack. You’re just layering AI over it, in the same way that dbt is currently layered over your data warehouse. The original data stack structure is still very solid, so we're just keeping that; AI now exists on top and covers the whole thing (or, alternatively, most of the original stack, because you absolutely can add this AI layer in pieces).

Where to start

The first step to implementing this new AI layer is to start with the highest-value layers. Here are places to consider adding in AI.

Code review is where you can get the most out of AI by integrating it for things like reviewing PRs, CI checks, flagging issues like 'Hey, this is missing. We need to take a look at it.' This is a natural starting point for introducing AI into data pipelines and workflows because it’s where developers are already using AI for their individual work.

Error triage is a logical place to layer in AI: making AI part of your code review process, using it for error tracking in your data transformations, and even in your data loading workflows. In practice this would look like setting up AI to do that initial triage work and then setting up alerting, be it through email, Slack, PagerDuty, or whatever system you use for error communications.

Extract/load is also a small but very targeted opportunity. Here, AI could be useful for evaluating what's coming in, making sure nothing coming in is messed up, evaluating errors, etc.

Your ticketing intake system can be another great place to start. One dbt customer created a ticket intake chatbot to make sure they had access to all relevant context around requests. This chat interface gathered information about the problem that they're trying to solve, rather than simply intaking a verbatim request that essentially dictates a solution. Business users often don't know what the data team has access to or even really grasp the whole breadth of what that team can do. Applying AI here to create collaboration can produce solutions that give more than the person requesting it even realizes is possible, which transforms a data team from order takers into internal consultants.

Ultimately, each layer of the stack from intake to data loading to data warehousing to consumption have use cases for AI. It's all about identifying those use cases and implementing them into that workflow, and you find them by looking for places where the addition of an AI tool would be helpful for you and your team overall.

A playbook for getting started

For an applied example, let’s say that you are currently the leader of a small data team of 10 people and you provide analytics to the sales team. You have a current modern data stack that consists of five ingestions, Snowflake as your data warehouse, dbt to do the transformations, and Tableau for your data visualizations.

A typical day-to-day for this team right now looks like this: business users put requests in, your data teams take those requests and create what's needed for the business, and finish with going through the review process. Where does AI enter into this picture?

Step 1: Find the first, low-stakes pain point

The first step in determining what you should do with AI is finding your use case. You do this by asking, 'Hey, where are our biggest pain points currently? Do we not have enough time to do extremely thorough code reviews? Or do we get incredibly vague information from business requests? Do we frequently get failed loads from our loading tool into the database due to changing APIs or other reasons?’

Code review is the example we will use here as it is what a lot of people start with to really get their feet under them because, if something goes wrong, there is typically already an assigned human in the loop. To do this, you would:

1. Start by deciding orchestration, figuring out what automations exist and how you want to run those automations. Are you going to just use something like GitHub Actions? Do you have an external orchestrator that will run this process whenever a pull request is created in your change control tool of choice?
2. Next, determine which AI company you want to use (which is often determined by whichever one(s) your company already has enterprise agreements with) and which of their AI models you want to use. Obviously, because it's your code, you want to make sure that there are agreements for security, for not using your data for training, etc. All this should be handled in tandem with the IT team and the security team.
3. Build the behavior as skills: You would put together a skill that says, 'Hey, you're a code reviewer. These are our coding standards. We don't use leading commas. We always use CTEs in this specific format, we're using the dbt standards.' Whatever is important to you, it all goes here. Pro tip: dbt has a curated collection of agent skills that help AI agents understand and execute dbt workflows more effectively.
4. Add the business context into the skill: You would also put information about the business, like the sales team expects this certain format, here's the basic structure of our data. You would build this all in markdown, use it as a skill that whatever AI tool you selected to automate this process can actually follow those instructions.

Step 2: Set up a pilot

Now, it’s pilot time for your AI-powered automated code review: roll it out, then let it run, monitor it closely, make sure you're still doing human code reviews. You’ll want to:

• Test and iterate before going wide. It all comes down to testing that automation. So either putting it on a small test repo or putting it out there, but only running it kind of ad hoc to make sure that it works. So, doing a couple of code reviews, giving it some code, refining what that skill looks like, refining what you're sending to the AI.
• Pilot small and expand outward. Your pilot is what builds that trust over time. Just releasing it to 500 people all at once will just create a mess trying to keep up with managing it.
• The final goal is up to you, but for most teams this will be putting the AI code review agent on your main repo so that whenever somebody pushes to that repo, they're going to get this automated code review.

Step 3: Gather feedback, create evals

Let your people who are excited do the actual work of setting the pilot itself. When it comes to evaluation, however, you need to have people from multiple walks of life do that review so that you can make sure you're getting a well-rounded understanding of what's working and what's not working.

The best feedback is going to come from a standard senior member on your team that is perhaps indifferent or maybe even a little combative when it comes to AI adoption. They'll be able to spot things that someone who's very excited about it may not spot. Have more senior members of the team who do a lot of code reviews also review the output and see if they can spot things that the AI is not doing correctly. This feedback can then be used to refine the Skill you are utilizing.

Once those more senior members of the team feel comfortable with the results that you’re getting, now you work with them to create the eval for keeping things accurate over time (we’ll talk more about evals shortly).

Finally: remember that this is all brand new, so don't feel like your pilot has to be perfect out of the gate. Making mistakes is how you learn not to make those mistakes; that's the IT way of life. The rite of passage for every data engineer out there is accidentally dropping a production table or breaking something in a transaction. But I predict that pretty soon another rite of passage for data engineers will be having an AI pilot that just didn't work out. Because, after all, how can you truly understand something unless you've broken it and then had to fix what you broke?

Maintenance and evals are part of the planning process

The crucial work in integrating AI into your data pipelines isn't actually building the pipelines, but making sure they stay reliable once they’re up and running. The most important message I can give you is that integrating AI isn't a one time effort. These systems need to be regularly evaluated by the team because LLM performance can degrade over time. Models change, something's being throttled, context window limits change, and sometimes the cost to performance ratio no longer makes sense.

Maintenance has to be a large part of the planning process, not an afterthought. If you don't think about maintaining an AI workflow from the moment you first decide that you're going to implement it, you'll end up with either something that nobody uses or something that degrades so much it erodes trust in the system.

Going back to our AI code reviewer example, most people set one up within probably a few days through GitHub Actions, a couple of API keys, a couple of calls, it's good, it does what you need. You’re not done, though. Where you're going to win or lose is how you maintain that system and how you make sure that that model is giving consistent results, otherwise all you're doing is creating noise that people won't use.

Evaluation loops are how you keep the system honest

An essential piece of integrated AI workflows is having evaluation loops where you can make sure that the LLM is still performing as expected in each one of these processes. Implementing these evaluation loops is one of the biggest pieces in productionizing AI.

In practice, evals look like different things depending on the workflow you’re attempting to automate. For an AI ticket intake chatbot, this could be having a generic ticketing workflow that you can automate to pass through the LLM, say once a week. For an automated error handling system, you’d send out a generic error message through your error workflow at a preset cadence so that it can be reviewed and you can make sure that the performance of the AI system you're using stays on point.

If you try to skip this part of the process, what can happen is people don't notice the degrading performance at first, until suddenly you're getting skewed data. Again, faulty data is really going to poison any kind of adoption.

Evals are OG ML

Evals aren’t something we’ve just added due to agents. People who've been in the traditional AI frameworks space largely understand that evals are an integral part of the machine learning flow.

Before I came to dbt I worked on a very large project doing predictive modeling for a manufacturing company; this was long before agents, but we still did evaluations as part of those workflows. One thing we would do for example is generate a month-over-month capture of basically, ‘Hey, we were doing good at predicting X last year. We're not doing so good now. Do we need to retrain or use a different model?’

A lot of people are coming into AI now because it's the new thing, and often they skip looking into things like evaluating and making sure it's a system that can stand the test of time in production. But it’s always been the case that, whenever you're dealing with a machine doing things on its own, you have to give it guardrails and give it controls, to make sure that you know what's going on under the hood. And I feel like that's missed by a lot of people in the AI race nowadays because everyone's just scrambling to even keep up with what's going on.

Evals run on humans, too

Ultimately, evals are only partly automatable. You could have automated systems that compare the output of your evaluation month over month, but you need a human in the loop for review. If there is any sign of degradation in performance you would want to have a human reviewer apply judgement: is this true degradation, or is the model just giving slightly different answers?

Other areas where humans still need to be involved in AI workflows include making decisions around whether to switch models or change something about the skills that you're using in your markdown files or changing what tools the agent can access through MCP. Changes like these need a human to review the results you’re currently getting, and make those determinations around any changes needed, and then rerun those evaluations to make sure that it's still giving the outputs that they would expect

My baseline message to builders here is, don't get lazy with it. Don't just accept whatever the AI says, or even use a majority of what the AI says unless it's very boilerplate. Make sure that these outputs are being reviewed, they're being vetted and holding up to the same standards that you've already set for yourself or your team. The advent of AI doesn't mean that we can now just relax on the work that we've been doing for years.

AI-ready data starts with leadership

Right now, AI is very fractured at companies, with different levels of adoption in different areas.

Every company wants to adopt AI, and I think most have moved a little too quickly. They give everyone some kind of AI tool and tell them to use it, but what's missing is actually working AI into their way of working. You think you've got AI in engineering because everybody's using Claude Code or Codex, but that's just the top level. That's like saying everybody uses Google at this point.

Making your organization’s data AI-ready means building AI into your workflows. Not just having your data engineers use AI tools, but also integrating them into your triage flow and your pull request flow and everything else your team does every day.

The more a data leader can work AI into the daily workloads of an organization, the more that will actually drive adoption in a meaningful way. This takes buy-in from leadership to say, “When someone submits a pull request, we want to have an automatic agent review. When there's a pipeline failure, we want to be able to see an automatic triage done by the AI system.”

For those of you in charge of actually building this, especially when you get those top-down “adopt AI now!” edicts from the C-level, VP level, people who may not be working with this technically — your challenge is communicating what it’s really going to be like.

The things your org’s leaders need to understand about AI

When it comes to fully integrating AI into your organization’s workflows, there are two important pieces of information that need to be communicated up the chain:

First, this is extremely exciting technology. But, because AI is non-deterministic by nature, making AI practical in the enterprise space means making it more deterministic. You achieve this through evals, and with aligning skills and giving clear instructions and guardrails to make it incrementally more deterministic. As a result, leadership needs to understand that, "Hey, we can't implement this tomorrow." It's like any other major software implementation; actually, it’s even more complex because you're dealing with non-deterministic LLMs.

Second, setting realistic expectations is key. Leadership needs to understand that velocity is not going to 10x overnight. Velocity will happen, but first the outputs need to be reviewed, the system needs to be trusted, evaluations need to be solid. Developers on the ground floor that are using these AI tools and seeing these outputs for things like error triage, code review, tickets coming in, and even text to SQL, will still need to review the outputs of the LLM.

Realistically, you can get efficiency gains with AI, but in these early days, I would say a lot of the efficiency gains you'll get with AI are going to be at least partially eaten up by the amount of work you need to put into making AI reliable and trustable.

Make AI organizational, not just individual

Using AI tools personally is great. It's fantastic, it's super cool, you're spending all your Claude credits, that's awesome. But if you really want to get the most out of your AI spend and the most out of your AI tooling, you need to work on productionizing it and building it into your daily tasks over time. That's the muscle we need to be building if we want AI to‌ make sense in our organizations going forward, rather than just as a personal tool.