From mainframe to AI-powered maturity: Verisk's data modernization journey

In a dbt Summit 2026 breakout session, Casey Schwegel, Vijay Singh, and Mateusz Geca, data engineers on Verisk Analytics' underwriting data team, told a 4-stage modernization story that will sound familiar to anyone who has inherited a legacy data stack: something that works, until it doesn't scale, followed by a fix that solves that problem and creates the next one.

Verisk aggregates insurance experience from hundreds of individual insurers, each translating its own systems into a required fixed-width format defined by a thousand-page PDF manual, a process the team admitted, dryly, rarely goes wrong. The output feeds actuaries, described as demanding data consumers whose entire job is turning that historical data into pricing decisions that get filed with state regulators. There is, in other words, no room in this pipeline for a bad number.

The data itself lives in tables with a few billion rows and a couple hundred columns each. For decades, that ran on a mainframe. "The mainframe wasn't broken," Casey said.

"It held decades of underwriting rules, and did its job exceptionally well." But tracing where a number actually came from, a rate-making rule buried somewhere in decades of undocumented logic, meant staring into a black box, and the team's own rule was: if a job fails, do not touch it.

The team's first move to the cloud, Snowflake paired with Jupyter notebooks, brought version control and visibility to business logic that used to be buried in mainframe job streams. It worked well for quarterly batch runs, but notebooks were never built to automate daily production. The next step, building directly on native Snowflake objects like streams, tasks, and dynamic tables, unlocked daily refreshes, but replaced the old problem with a new one: hundreds of isolated tasks and tables that had to be manually coordinated, with no way to promote code cleanly between environments.

Tables and views stored data and logic, materialized views sped up the heaviest datasets, streams and tasks captured change data on cron schedules, and dynamic tables refreshed automatically against a target lag. It worked, but maintaining it meant babysitting hundreds of individual objects by hand: an upstream schema change or a failed task meant combing through query history one entry at a time.

dbt is what finally unified all of it: version-controlled models with peer review, self-documenting lineage instead of stale wikis, and tests that halt a pipeline before bad data reaches an actuary rather than silently corrupting a report. Because Snowflake allows identical table names across different schemas but dbt requires every model name to be unique, the team organized its project by business line, with staging, intermediate, and mart layers nested inside each, and adopted a naming convention combining layer, domain, and object name so nothing collides. A custom macro reads a marker in each model's file path to resolve where it should land: a developer's personal schema in dev, a shared schema in production, no manual aliasing required.

Before dbt, the team was contending with three different sources of truth that never quite agreed: production logic that got patched directly in Snowflake, source code sitting in Git that lagged behind, and documentation living in Confluence that stopped reflecting reality months earlier. dbt's doc blocks collapsed that into a description written once in a markdown file that gets referenced everywhere it's needed, so updating it in one place updates it everywhere automatically, version-controlled and reviewed alongside the code instead of drifting apart from it.

As Mateusz put it, walking through the implementation, "this talk is not really about dbt at all. It's more about the changes that happen after adopting it."

That governed foundation is now what's letting the team layer on AI: propagating documentation across the DAG with a custom dbt Wizard skill, using project context to get better AI-generated SQL, enforcing standards automatically on pull requests, and cross-referencing job logs with query history to speed up debugging. Documentation was the first target. A dedicated dbt Wizard skill combined with the doc blocks pattern now propagates column descriptions down through the DAG automatically once a developer writes the initial source descriptions, rather than copying and pasting them by hand.

That same layer of rich context makes AI-assisted model development more reliable, since a request can be framed in business terms instead of just raw SQL mechanics, and a separate skill encodes the team's own standards, naming conventions, required YAML fields, so a developer can run it against their own pull request before anyone else even looks at it.

All of it, the team was careful to note, stays human in the loop. "AI is going to thrive on patterns and context," Casey said, "and the better our documentation, naming, lineage, and standards became, the more context the AI had to work with."

Audience questions pushed on the human side of this transition: onboarding staff with no Git or SQL background, and the team's honest answer was that some people just have to lead the shift and bring everyone else along, since the direction is only getting more technical. On the data side, the unresolved challenge is scale: actuaries want ten-plus years of fully updated historical data, and incremental models are still catching up to that ask.

Watch every keynote and breakout from dbt Summit 2026 on demand. Watch dbt Summit 2026 on demand →