Tom Nagengast is currently an analytics engineer on Netlify’s data team. Thanks Tom for sharing your story!
TL;DR: Tom’s career path, which he’d describe as “roundabout,” like many of ours in analytics, has flowed from:
Take it away Tom!
I took a pretty roundabout path to get into analytics.
I actually grew up in the wine industry, so I thought that’s where I was going. I wanted to be a winemaker and do that whole ag side of the world.
My brothers and I were always into tech, so after I graduated college I did some travelling and started doing web consulting work on the side. Mostly WordPress deployment type projects.
Then I started getting jobs working at wineries. I did a few harvests down in New Zealand and here in California. But eventually I realized that wine wasn’t really for me, in terms of a profession.
When I got back from New Zealand, I was just looking for a job that would be a good fit for me.
I wanted to work in web development and web design, but I wasn’t classically trained so I ran into a lot of dead ends.
I finally got a job as a sales rep at Mindbody, which is based in San Luis Obispo, and I went from sales almost directly into analytics. I found myself spending more time doing analysis on who would be the best person to talk to, instead of just going out and talking to those people.
So I went into analytics and started working on the Mindbody Sales Operations team.
It was pretty new at the time, and we were trying to figure out how to handle these massive spreadsheets that we were using for our monthly and quarterly reporting, and also how to handle automation for a large global sales team.
A big part of my focus was trying to get sales analytics setup for the company, which was a really exciting challenge.
We didn’t have any data engineer support in sales at the time, so we were just kinda cowboy’ing through all the different problems that we had.
We set up a good system, but around 2016 we went from having a lot of distributed analytics teams, to bringing them all under a Data Science org.
That’s when we had analysts from customer service, from product, from sales and marketing, all come together to form that central hub of analytics.
And that’s really where analytics took off for me, because I saw what was possible across the company.
Data science was still really new and mostly gaining traction at larger tech companies working with text and vision.
I had heard of data engineering at that point, but still didn’t really know what they did. There was no such thing as an Analytics Engineer at the time.
So I was thinking, what can I take from this web development/software engineering world, and apply it to the problems that all of these analysts are running into?
I was trying to come up with solutions to problems like moving data sources from a staging environment to a production environment and scaling the analytics we were working on.
Before dbt, we had our queries that were just stored in the database and you’d have to copy the DDL and re-run them to make any updates. Everything was directly in the database.
We realized that if the database goes down, or something gets dropped with a cascade, then you’re going to lose a lot of your work.
So we started storing them in this shared repository with just a ton of SQL files.
There was no orchestration between them. We’d have to read through our SQL and understand, this model relates to this one. And if this one fails, you need to update it, then copy this code and bring it into your editor and work with it there. It was better than having the source in the database, but far from great.
I was doing a bunch of searching on how other people were solving this problem.
That’s when I saw what they were doing with dbt and it looked really exciting.
We were on a SQL Server stack at the time, which was difficult to get dbt to run on (editor’s note: SQL Server is now supported via a dbt community adapter, though it wasn’t at this point).
So I pitched it to the team to see if we could do some more exploration on it, and it just didn’t quite work yet.
But during that discovery, I was introduced to the dbt community. I got into reading all of the dbt blog posts and joined the Slack group.
This was over a year throughout 2017 and into 2018, of just kind of watching the community. I didn’t get very involved though.
I was more in the background - like, what are they doing over there? It looked so cool.
And then around the end of 2018 and into 2019, we started moving from SQL Server over to Redshift.
I knew dbt supported Redshift out of the box and I decided I was going to start trying dbt just personally.
So I had all my stuff going with a little personal dbt project, trying to iron out all the kinks that I might have in my personal workflow.
Then once I had a stable setup, I could show the rest of the team and see if we can get more adoption.
My programming background was from the Laravel world and they were early adopters of Vue. So I understood Webpack and build processes where you define all your config and just say “run” and it handles everything for you.
I was trying to think of ways to take this automated system and bring that over into the antiquated SQL world.
Fishtown was doing the same thing with dbt and I thought, okay, this – this is going to be where analytics heads.
The build processes, the version control, the testing and documentation — how do we keep all of our queries in one spot so that we can collaborate on them?
If I create a model, how can another team member come in and add onto it without having to copy it, rebuild it with a whole new name and add their initials as prefixes?
Let’s have a single source of truth that everybody can contribute to and collaborate on. No more fragmented versions of models duplicated across the database.
In data science, you’re often working at the lowest level of the data. Sometimes bringing in custom data sources for an experiment or analysis, and writing a lot of SQL, python, R, VBA… whatever gets the job done.
In analytics engineering, you’re not doing as much presenting, but you’re still doing a lot of that dataset creation, building pipelines, and introducing engineering best practices.
It was a lot of the different parts that I was finding myself really passionate about, combined into one role.
I was in dbt Slack, and I saw that Emilie [Schario] had posted an opening for an analytics engineer.
At first I thought – what even is an analytics engineer? So I started looking and reading.
I think there are a bunch of blog posts now, but there were only a few back then. As I read them I saw all the things I liked about what I did.
And when I went to look at the job posting, Emillie had a 30 / 60 / 90 day breakdown of what was expected. I thought it was so clear - I’d never seen it laid out quite like that before.
I understood the vision, I saw what she was trying to do, and I really wanted to be a part of it.
My family still has our small winery here in Paso Robles, and one of the fun projects I’m playing with is bringing analytics into our harvests.
For example, with the way wine works, after the grapes are picked, they’re typically fermented over a two week time span and you can have all sorts of things go wrong during that fermentation process.
So I’ve been thinking about how to get that data and plug it into some kind of model where we can then predict which direction the temperature, sugars, acids, etc. are going to go. Or when can we expect a ferment to finish so we can open up the tanks and keep things moving?
Another more recent project was looking at how exercise affects my daily output. Does taking the time to move around make me more or less effective professionally or creatively? It’s a fun one to track because I’ve always seen exercise as a means to an end and is the first thing that gets pushed to hit a deadline.
To be able to have the experience to decide, alright we’re going to pull this data — we’re going to clean it up and transform it with dbt. We’re going to put it into whatever ML products that we’re using or just play around with it in Jupiter or in Python. That’s a lot of fun for me and gives me ways to try out new tools.
We love what we do, and I want to tell everybody about it.
I’ll talk to my family or friends in other industries and try to tell them about what I do or what I’m working on and I think the most common response is “You’ve lost me” coupled with a blank stare. So it’s difficult.
For a long time, I stopped trying altogether. But I’ve recently started talking in more general terms and explaining how we help people answer hard questions, rather than specifically what’s involved.
We bring data together and produce analytics that help people make decisions at the company.
Sometimes that’s still hard for people to understand though.
I was talking to a older family member, and they responded with something like:
“When I was your age, we used to build things that you could hold at the end of the day.”
I said, “we’re still building really cool, helpful things!”