Revamped dbt Semantic Layer: Enhanced metrics definition and querying with MetricFlow from Coalesce 2023

"The Semantic Layer is ready. What does that mean? Well, it means that you should go play around with it. You should test it out. You should connect it to your data applications."

- Nick Handel, director of product management at dbt Labs

dbt Labs’ Nick Handel, director of product management, and Roxi Pourzand, product manager, introduce the new dbt Semantic Layer. They highlight key features and potential future developments of this tool and cover how it can be integrated with various platforms.

dbt Labs has launched a new semantic layer to improve data analytics workflows

dbt Labs has released the new dbt Semantic Layer, a tool designed to make data analytics workflows faster, less error-prone, and more efficient. The dbt Semantic Layer was developed to address common issues such as errors due to complex SQL queries and poor governance. It’s also designed to provide consistent definitions, improve discoverability, and offer cleaner logic.

"Analysts are taking queries. They're pulling from a bunch of different things they've written before, they're modifying them slightly, and it takes time to get to the correct answer," explains Nick. He pointed out that the dbt Semantic Layer helps to reduce these issues by providing consistent definitions and improving discoverability.

Roxi adds, "This is one of the key, foundational integration pieces between our products post-acquisition–bringing in the best of what MetricFlow had to offer with the best of what dbt Labs had to offer." She also highlights the importance of the Semantic Layer's join support, which enables users to define relationships between datasets easily, leading to improved reusability and discoverability.

The dbt Semantic Layer supports a broad range of data applications

The dbt Semantic Layer supports a wide range of data applications, aiming to push up the hierarchy of data analytics needs and allow users to maximize their potential. Expressiveness, performance, connectivity, security, context richness, and engineering workflows are the six key pillars of dbt Labs' vision for dbt Semantic Layer.

Nick stresses, "Semantic layers map data to language. Data is used by computers and analysts. Language is used by humans." He adds that dbt Semantic Layer enables a more streamlined process of turning data into meaningful language.

"The goal of dbt Semantic Layer is to push us up and allow us to achieve the top of that hierarchy of needs…not just AI and deep learning, but metaphorically, the top of all of the things that we could do with data," he says.

Adoption of dbt Semantic Layer comes with several features

The dbt Semantic Layer is now available for general use, and it comes with a host of features. These include defining semantic objects on top of dbt models, support for complex metrics, GraphQL and JDBC APIs, support for additional data platforms, clean and legible SQL generation, and integrations with various data tools such as Tableau, Hex, Mode, Lightdash, and Google Sheets.

"You have a semantic model, which is a new concept we introduced, and it's referencing a dbt model here, and this concept of semantic models is what allows us to enable joins," explains Roxi. "We've also released Google Sheets and Tableau, as well as a partner integration of Lightdash."

Nick adds that dbt Semantic Layer's future includes features like caching, exports, and saved queries, more advanced permissions and security capabilities, additional integrations, and more metric types, hinting at dbt Labs' commitment to continual improvement and expansion of the the dbt Semantic Layer.

Insights surfaced

• The dbt Semantic Layer allows for the definition of semantic objects on top of dbt models, enabling joins and making code more reusable
• The tool can create complex metrics with a simpler interface, reducing the complexity of SQL queries
• The Semantic Layer offers interfaces for developers and consumers to define and consume metrics wherever they are, with support for additional data platforms like Snowflake, BigQuery, Redshift, and Databricks
• The tool is designed with a focus on trust, aiming to provide accurate, consistent data across various platforms
• Future developments for the tool include caching, exports and saved queries, advanced permissions and security capabilities, more integrations, and more metric types