Getting the most out of your dbt Cloud deployment

When it comes to any software investment, the ultimate measures of success are whether it helps you do your job better, drives your organization toward its goals, and perhaps even gives your career a boost. This year at Coalesce, I got so much energy from hearing our customers talk about the impact that dbt has had on their lives. We even had a wall at our booth dedicated to stories from dbt users.

Certainly, the topic of how to maximize success and get to ROI quickly is resonant and timely as data organizations look to justify budgets and improve their standing as strategic partners to the business. This also happened to be the focus of my Coalesce presentation, which you can watch in its entirety right here.

Below are a few tips on how to maximize the ROI of your dbt Cloud deployment, with a focus on onboarding additional users and teams, and then enabling those teams to provide a performant, governed data delivery experience.

Laying the foundation for success

The first step is to define what successful data delivery looks like for your organization. Who are your stakeholders? How frequently does the data need to be updated? Are there certain SLAs for data freshness and quality? Defining the success criteria will help tailor your approach to using dbt Cloud effectively.

Next, think about your organizational structure. Are you a smaller team managing all analytics centrally, or are you planning to scale up and empower multiple teams to build and maintain their own data models? Your deployment strategy should align with your team composition and growth plans.

Onboarding teams efficiently

Efficient onboarding is critical for a good ROI. The faster new teams or users can start using dbt Cloud to deliver data products, the quicker your organization will see value.

We’ve all experienced slow, painful onboarding processes. What should take a few days often stretches into weeks because of issues with setting up access and configuring data connections. With dbt Cloud features like role-based access control, single sign-on, and global connections, you can streamline onboarding users.

Streamlined access and connections

To onboard users quickly with the right access controls, start by implementing single sign-on (SSO) and role-based access control. This ensures that users can quickly and securely access dbt Cloud with the appropriate permissions.

Recommended implementation:

1. Set up groups in the identity provider that map to dbt Cloud groups.
2. Use SSO so users can log in seamlessly and group mapping to ensure users are added to the mapped group in dbt Cloud.
3. Assign groups access to projects using pre-defined permission sets.
4. Designate project admins to manage access for their teams, reducing bottlenecks and administrative overhead.

Efficient data platform connections

Configuring data platform connections manually for each project can be time-consuming and error-prone. Instead, use global connections in dbt Cloud. These connections can be reused across projects, allowing teams to self-serve their setup and reducing duplicate work. For developers, consider using external OAuth authentication or native warehouse authentication. This will help them easily connect to data platforms without managing credentials manually.

Key takeaways for onboarding:

• Use SSO and mapped groups for efficient and secure access.
• Set up global connections at the account level to reduce redundancy.
• Implement OAuth to simplify developer authentication.

Delivering governed and efficient data products

Once teams are onboarded, the next step is to enable those teams to build high-quality and up-to-date data products, while optimizing the resources required to do that efficiently.

With multiple teams contributing, it's crucial to have visibility into how they’re using dbt Cloud and whether they’re following best practices. The Data Delivery Insights dashboard (beta) surfaces metrics like job performance, test coverage, and source freshness to give you this visibility.

Here are some ways you can use these metrics to empower teams with the visibility they need to deliver consistent, high-quality data assets.

• Test coverage: Use the dashboard to check if models have adequate tests. If coverage is low, encourage teams to add more tests to ensure data reliability.
• Models built: Understand how active a project is based on how many models have been built in the last 90 days
• Source freshness: Review sources and flag any stale data sources with the appropriate teams.

Data Delivery Insights is currently in beta—if you’re interested in joining the beta, reach out to your account team.

Optimizing for efficiency and quality

Once you have visibility into the reliability and efficiency of your data delivery workflows, use dbt Cloud features like Explorer and micro-batch incremental models to improve those metrics.

1. Explorer: Improve data quality and accessibility by using lineage with the new query history lens to audit and document frequently queried models while removing or consolidating underused ones.
2. Micro-batch incremental models: Reduce average model build time with microbatch incremental models. While incremental models allow you to limit the data that needs to be transformed to only the rows that have been created or updated since the last run, microbatch allows you to break up large datasets into smaller batches to make updates faster and more manageable.

Conclusion

You can maximize your dbt Cloud deployment by onboarding teams more efficiently and supporting those teams to provide high-quality, governed data products. Scale onboarding and deliver data faster with functionality like SSO, role-based access control, and global connections. Get visibility into the health of data delivery across your organization, and build a strategy for optimization powered by dbt features like microbatch incremental models and Explorer.

If you’re interested in being a design partner or getting access to beta features, reach out to your account team. We’d love to hear your feedback and help you make the most of dbt Cloud.