dbt Labs + Fivetran: Open data infrastructure for analytics and AI

This is unlike any anything I’ve ever written. As I sit down to write, there are just so many emotions. I’ll reflect on some of this later in this post, but for now, I won’t bury the headline.

Today we are announcing that we are merging with Fivetran.

The merged company combines for ~$600 million in ARR with well north of 10,000 customers. The majority of companies who use the cloud for their data infrastructure use one or both of these products already.

This kind of scale and reach is surreal to think about, and it brings me much closer to my own personal goal of the past 6 years: to create the organization that can invest in and steward dbt and its community over the very long term. Behind the numbers though, the real story here is what this means for the dbt Community, for our customers, and for the future of the industry. That’s what I want to spend most of this post on.

But I want to start with the motivation for the merger and why I’m personally so excited about it.

The backstory

I’ve known George and Taylor, the two co-founders of Fivetran, for a decade. Starting Fishtown Analytics back in 2016, Fivetran was one of our best partners. Our stack at the outset was pretty much always Redshift + Fivetran/Stitch + Looker/Mode. We brought Fivetran dozens of customers from 2016-2019. For a while, Taylor was our partner manager. I filed support tickets and asked for new connectors. As a partner, Fivetran was never anything less than fantastic.

Then came the modern data stack buzz era. Both companies competed to out-fundraise each other for a couple of years ;) George and I were on a million panels together, and every single conference I went to I would always wander over to the Fivetran booth to catch up with Taylor and George.

I think it’s fair to say that no two companies in the modern data stack have been more joined at the hip. The products were literally built to be used together. The companies have grown up together, the relationships at all levels of both companies are longstanding, and the respect is mutual.

It has even seemed obvious for a while that it made sense to merge—the questions were really more about when and how to do it, and whether we could figure out how to make it work. I remember a conversation about this with Martin Casado at a16z, who is on the board of both companies, a few years ago. His response: “You and George have to figure that out for yourselves, but from my perspective it is just such an obvious win for everyone involved.”

I agree—this has been a good idea for a while. And the answer to “why now” is really about maturity. Both companies have reached a stage where we have a level of predictability around our existing businesses, and we were both thinking about what was next. We decided that, as we traveled that road ahead, we’d be far better positioned to do it together.

What we can build—together

Right out of the gate, let me just say something basic, but important. dbt will still be dbt. Fivetran will still be Fivetran. We’re not planning to rename either product, not planning any disruptive product changes, we’ll continue to provide the same types of support for the dbt Community, and are aggressively executing against our respective product roadmaps.

In short: if you use and love either product today, this merger will be non-disruptive.

The more interesting question, really, is: what can we now build together that we could not build alone? This is where I get really excited. And it really all comes down to two principles that both products hold dearly: simplicity and openness.

Simplicity

From the beginning, dbt and Fivetran have both attempted to make the sometimes-arcane practice of data engineering simpler. Fivetran’s integrations have always been push-button: flip a switch and get data, reliably, with low latency. dbt’s opinionated programming model allows less technical data practitioners to author production-grade data pipelines.

George likes to think about this like electricity: flip a switch, get data. I like to think about it like the Apple ecosystem: it all just works together, no duct tape. We both agree that far too much time is wasted in data engineering toil that creates zero business value, and we’ve built products that make the work of data engineering simpler, more automated, and more accessible.

Openness

Both products have also always been focused on being open and interoperable. That is to say, allowing users to build data pipelines that work with any underlying data platform. Both dbt and Fivetran abstract away the details of the underlying data platform and the cloud it runs on, allowing users greater strategic flexibility with one of your most core IT assets.

There was a time when this open pattern was almost assumed, but as each of the data platforms has begun to invest more heavily in building their own first-party tooling, this trait becomes ever-more-important.

Open data infrastructure

Many users think of Fivetran as ingestion and dbt as transformation, but both products have grown meaningfully over the last several years and at this point cover a very significant footprint. The combination will form the most complete, most widely-deployed open data infrastructure platform on the market.

Here’s what it looks like when both products come together:

This architecture is what we are calling “open data infrastructure.” Open data infrastructure describes an infrastructure that is pluggable, relies on integration via standards, does not assume the usage of any one particular compute engine, and does not assume that solutions will be duct taped together from many individual products and vendors. It is more integrated than the modern data stack, and it allows for greater user choice than the all-in-one data platforms.

I’ve written a whole lot more about open data infrastructure and the journey that our ecosystem has been on over the past several years here. And if you’re interested in hearing more about why I think this is so critical to the future of analytics and AI, tune into my Coalesce keynote tomorrow.

Open source

If you are a dbt user, what you will likely want to know is: how does this impact dbt open source projects—most importantly, Core and Fusion?

You can read George’s own post on this topic, but part of what made this merger such an obviously good idea to me was Fivetran’s complete alignment in how it sees this. Our collective commitments are as follows:

• dbt Core and Fusion will both continue to be shipped under their current licenses.
• We will continue to actively maintain dbt Core.
• We will continue to support the dbt Community, via Slack, meetups, etc.

To give credit where credit is absolutely due: Fivetran has been a long-time contributor to the dbt open source ecosystem, authoring over 100 packages with OSS licenses that are used by thousands of teams. They have 5 full time staff now dedicated to this work. I think that the extent of their contributions here over the years has gone under-appreciated; it’s likely that they have contributed more OSS dbt code than any other organization outside of dbt Labs.

I know that at least some of you reading this will be concerned that Fivetran’s historical focus on building proprietary software might make its way into the ethos of dbt, but we anticipate quite the opposite. Instead, George and I are excited to bring more openness to the Fivetran ecosystem with the combined company. We are already thinking about what existing software can be shipped under an OSS license (connectors? connector development kit?) and what standards we can invest in (or create).

While we don’t know all of the answers yet, we absolutely believe that open source is a win-win for the community and the company, and we anticipate building more, not less, of it. Figuring out shared OSS strategy will be one of my biggest charters in the combined company. My title will be Co-Founder and President, I’ll have a seat on the board, and I’ll be responsible for our community and open source strategy. So: I have every intention of continuing our decade-long trajectory of innovation in the open.

So much more to do

I mentioned at the beginning that there are a lot of feelings swirling around as I write. This is true. A lot of gratitude for a lot of people. Some natural stress associated with any big change. Maybe a little bit of nostalgia hiding in there somewhere :) And a few things that I’ll probably have to work out with my coach!

But this is not a freakin’ Oscar speech. This thing isn’t done. We have more work to do, I have more work to do. Open standards and AI both completely rewrite the script of how analytics is done, and I’m excited to step into this new future. The combined company is well-positioned for this future—fine—but for the moment I just want to say that I am excited about this moment as a data practitioner. There is so much to figure out, so much to play with.

In the 2010’s we got fast analytical databases in the cloud and we got to work a lot more like software engineers. In the 2020’s we’re getting heterogeneous analytical compute and computers that can think. The world of the data practitioner will just be so much different in 2030.

The best practices for this next era have not been figured out. We don’t yet know what’s possible. And that’s when it’s fun.

Today feels like things felt back in 2015-2017; we get to play with these cool new toys that are starting to work and figuring out what we can do with them. I am excited to get my hands dirty and figure it out right alongside of you. And I plan on sharing, in public, throughout that journey.

As always, do not hesitate to ping me on dbt Slack—I’m @tristan.