How Ramp uses dbt and Materialize for operational data warehousing from Coalesce 2023

"A batch analytical data warehouse like Snowflake is not sufficient for an operational workload like fraud…where you need to respond to a new fact coming into your business in minutes or seconds…"

- Nikhil Benesch, CTO of Materialize

Nikhil Benesch, CTO of Materialize, and Ryan Delgado, Staff Software Engineer at Ramp, discuss how Ramp, a finance automation platform, uses Materialize and dbt to combat fraud. Ryan explains how Ramp’s team built their fraud detection platform, the challenges they faced with latency and cost using Snowflake, and how they migrated to Materialize to solve these issues. Ryan also covers the advantages of using Materialize with dbt and the future plans for their system.

The integration of Materialize with dbt significantly reduces latency and improves data freshness

Ramp integrated Materialize with dbt to tackle fraud detection. Their previous system was based on batch analytical infrastructure with significant delays in data freshness. Ryan explains, "When fraud happens, it happens really quickly... So that means that speed is really really key…to minimize our losses from fraud." Materialize provided them with real-time computation of data, reducing end-to-end latency for feature computation from an hour to one to three seconds.

Nikhil adds, "Materialize handles it all in real-time... What used to be a slow, 30-minute batch, from here to here, instantly flows into Materialize where it gets computed with the same SQL queries that you're used to writing."

Materialize can significantly cut down costs and improve speed in data operations

In addition to reducing latency and improving data freshness, Materialize helped cut down costs. Previously, Ramp's cost for using Snowflake was ballooning to about $120k per year. However, by integrating Materialize, they managed to make considerable savings while improving speed.

Ryan explains, "We refactored [our] fraud detection system to be based on Materialize rather than Snowflake…We actually had a 60% reduction in total monetary loss rate…that's a lot of money, so definitely justified our spend on Materialize and also our engineering effort there."

Materialize enables real-time data applications and improves business outcomes

The integration with Materialize also led to improved business outcomes for Ramp. They reported a significant reduction in fraudulent transactions due to the platform's real-time data computation ability.

Ryans elaborates, "50% of hacked accounts were flagged as fraud before fraudsters could spend any money–something we couldn't do before…”

Nikhil emphasizes the value of Materialize for real-time data applications, stating, "We built Materialize, the world's first operational data warehouse… you can build real-time data applications right on top of Materialize with very little code of your own in the way."

Nikhil and Ryan's key insights

• Ramp's fraud detection system was initially on Snowflake, but they faced issues with data freshness and high costs
• Materialize offers real-time data updates, making it a more efficient tool for operational workloads like fraud detection
• Materialize pairs well with dbt, allowing for continuous computation of models and instant updates
• Using Materialize, Ramp was able to reduce the latency of their feature computation from every hour to one to three seconds
• The integration of Materialize with dbt allows not just data engineers but also analytics engineers and fraud analysts to develop these pipelines using SQL