State of Analytics Engineering
A survey of pains, gains, and areas of investment for global data teams.
Benchmarking the analytics engineering organization.
In January of 2019, Michael Kaminsky, writing for Locally Optimistic, hit publish on a blog post titled The Analytics Engineer. The blog post described how new tools were enabling any analyst who knew SQL to manage code bases and contribute to the data development process. The blog post perfectly named an experience that many data practitioners were having at the time.
Two years later, Seth Rosen infamously claimed: An analytics engineer is really just a pissed off data analyst. And 18 months after that, analytics engineers made the mainstream press. Business Insider covered the rising salaries of this emerging role, stating that companies like Amazon and Apple were hiring for the desirable skill-set.
All hype and hot-takes aside, analytics engineering simply describes how modern tooling allows data practitioners to apply software engineering best practices to the data development workflow. Is it a title? Sometimes! But more importantly, it’s a workflow that allows analysts, data engineers, data scientists, and really, anyone who knows SQL, to collaborate on data sets.
In an effort to bring more transparency to this practice, dbt Labs is proud to introduce the results of our first annual analytics engineering survey. Below you’ll learn how data practitioners compare in their goals, initiatives, and areas of investment, and how this information differs by features like organization size, sector, and structure.
What did we learn?
In addition to the highlights below, we’ve made all 567 survey responses publicly available. We encourage you to continue exploring the segments that best match your situation, to better inform how you apply these insights in your role.
46% of respondents plan to invest more in data quality and observability this year— the most popular area for future investment.
Lack of coordination between data producers and data consumers is perceived by all respondents to be this year’s top threat to the ecosystem.
Data engineers and analytics engineers are most likely to believe they have clear goals, and most likely to agree their work is valued.
71% of respondents rated data team productivity and agility positively, while data ownership ranked as a top concern for most.
Analytics leaders are most concerned with stakeholder needs. 42% say their top concern is “Data isn’t where business users need it.”
Thanks to our collaborating partners, who provided feedback and promotional support during survey development and recruitment:
Jobs to be Done
Data titles are notoriously poor indicators of jobs-to-be-done—one team’s Data Analyst is another team’s Data Scientist. This inconsistency can leave practitioners feeling alone in challenges faced. What if we looked for community in how work is done—rather than by whom.
As a percentage of your company, how many people are responsible for data infrastructure?
Data infrastructure teams — responsible for maintaining the platforms for hosting, moving, and accessing data — make up 5% or less of total company headcount, for the majority of our response set.
Data infrastructure team size
As a percentage of your company, how many people are responsible for data analytics?
Data analytics teams — responsible for building and analyzing data sets and reports — make up 5% or less of total company headcount, for most in our response set.
Data analytics team size
What determines how you distribute work within your data team?
Should data teams be centralized or embedded? Does it depend on the role, the company size, or something else?
Overall
By title
Data Engineers more often align by function, while Analysts and Analytics Engineers tend to embed in business areas.
Which of the following best describes how you spend most of your time?
Most respondents devote most of their working hours to building or maintaining datasets for analysis.
Overall
By title
While distribution of time spent preparing data differs by role, almost every role spends most of their time here.
Compensation
The modern data stack changed data team structures. New roles were born, and others evolved. dbt has been a catalyst for both—blurring the line of responsibility between traditional data engineers and data analysts. But has a shift been fairly reflected in compensation?
What is your annual salary (in USD)?
On the whole, Analytics Engineers and Data Engineers tend to make more than Data Analysts.
North America
Most Analytics Engineers (87%) and Data Engineers (68%) in North America make more than $100,000 in base compensation. Less than half of all Data Analysts in this region make more than $100,000.
Europe
Compared to peers in North America and APAC, practitioners based in Europe are less likely to earn six figures until they reach Manager or Director-level positions.
Asia Pacific
53% of the Analytics Engineer respondents in APAC countries make over $100,000, while 25% of Data Analysts and 20% of Data Engineers from this region have reached the same compensation level.
Challenges in Data Preparation
Regardless of title, most survey participants cited spending the majority of their time preparing data for analysis. This may be due to the scope of work, or the lag in advancement and lack of standardization across tooling focused on this phase of work.
What do you find most challenging while preparing data for analysis? Select your top three.
Unclear data ownership and poor source data quality are the top-ranked data preparation pain points for all participants.
Overall
By title
Data Engineering leads are more likely to struggle with documentation and dependency management while Analytics leads face unclear data ownership.
Perception of Performance
“How’s my driving?” Your data team works hard to hit delivery deadlines, iterate on output, and answer “Are you sure this is right?” at all hours of the night. But does all that effort accrue to anything? Do stakeholders agree? How do you know what success really looks like?
Rate your organization across the following areas.
We asked survey respondents to rate their organizations across several types of data work, from analytics to self-service enablement. Over half score their company well across every category — with “data team productivity and agility” receiving positive scores from the largest portion (71%) of respondents. “Cross-team alignment on data ownership” is bottom-ranked, with the highest percentage (44%) of people reporting poor performance in this area.
Overall
Marketing Analytics
Product Analytics
Financial Analytics
Data Quality / Transparency
Cross-team Alignment on Data Ownership
Data Team Productivity & Agility
Enabling Self-Serve Analytics
Success Metrics
Data teams help every part of the org measure things, but often struggle to align on how to best measure their own impact. Time-to-delivery is easy to quantify and easy to track. But what about time to decision? Or value of decisions made without data doubt?
How does your team primarily measure success?
Over half of respondents define success by relationships with others, and 29% rely on evaluations from stakeholders.
Overall
By title
Do you agree with this statement: 'My organization sets clear goals for the data team. We have a roadmap on how to execute.'
Analytics Engineers and Data Engineers have the brightest view of the data team’s responsibilities and roadmap.
Overall
By title
Do you agree with this statement: 'My organization values the data team. We are respected and included in decisions that impact our work.'
Managers are more likely to say that the organization doesn’t value their team, but ICs don’t often agree.
Overall
By title
What’s Left to Solve?
The modern data stack is overflowing with solutions for every part of the pipeline. It’s easier than ever before to move, store, transform and explore. But what products will actually solve the people and process problems behind even the most modern of modern data stacks?
What are the biggest problems still facing the modern data stack?
Companies large and small still struggle with coordination, hand-offs, and ownership. Are data contracts the answer?
Overall
Areas of Investment
The current macroeconomic climate has forced many organizations to rethink their financial strategy. Data teams are being asked to do the same—but can’t afford to reduce investment in areas that help their business make smarter decisions with stricter budgets.
How have recent changes to the macroeconomic environment affected your data team?
While larger companies are more likely to reduce headcount, they’re also more likely to invest in tooling to fill the gap.
Overall
How is your team thinking about investment in the following areas over the next 12 months?
46% of respondents plan to invest more in data quality and observability—the most popular area for future investment.
Overall
Organizations with 501+ employees
Organizations with ≤500 employees
Survey Participants
The dbt Community is home to 50,000+ data practitioners—making it an incredible resource for a variety of data roles and respective areas of focus. Not all participants have extensive (or any!) experience in dbt, but all share a passion for the practice of analytics engineering.
What is your job title?
40% of survey respondents carry Analytics Engineer or Data Engineer titles, and 20% identified as a manager or lead.
Overall
Approximately how many people work at your organization?
Survey participants come from organizations of all sizes— with half at organizations over 500 and half below.
Company size
Where is your company headquartered?
95% of responses cited company residence in North America, Europe, or Asia Pacific.
Region
What best describes your company industry?
Technology, Consulting, and Financial Services are the 3 most-represented industries, accounting for 59% of all respondents.
Industry
Does your organization use dbt Core or dbt Cloud today?
76% of respondents work for companies using dbt, with open-source users making up the largest segment.
dbt deployment
How long have you personally been using dbt?
Half of respondents reported 1 or more years of hands-on experience with dbt Core, or dbt Cloud.
Overall
Methodology
- This report is based on a survey of 567 data practitioners worldwide, conducted between October-November 2022.
- The average participant completed 64% of the questions, taking an average of 7 minutes. No material incentives were offered for taking the survey.
- The survey consisted of 20 questions, 19 of which were multiple-choice.
- Why don’t the totals add up to 100 (percent) or 567 (total responses)?
- Many questions invited respondents to select multiple options (e.g. “all that apply” or “top three”); in those cases, data visualizations may show percentages that add up to more than 100%.
- Where we segment by job title — we've focused on the 78% of respondents who have data-focused practitioner or leadership titles — leaving out those who skipped the title question or provided non-data titles.
- Where we segment by company size — we've left out those who skipped the company size question or said they were unsure how to answer.
- 82% of respondents were recruited via channels owned or operated by dbt Labs. Top sources were opt-in emails, promotional placements on getdbt.com, and both physical and digital signage at the 2022 Coalesce conference.