Using dbt for multi-project orchestration in surgical robotics and analytics engineering from Coalesce 2023

"So that journey from solos, or should I say silos, to symphony is the essence of our data journey."

- Christopher Nourse, Analytics Engineer at Intuitive Surgical

Team members from Intuitive Surgical, Brice Loustau, Software Engineer, and Christopher Nourse, Analytics Engineer, describe the data journey of Intuitive Surgical, a company that develops, manufactures, and markets robotic products. The session focuses on how the company uses dbt to manage its data and the challenges it faces in a highly regulated industry.

The journey of data management from individual silos to a harmonious symphony

Christoper and Brice draw an analogy between data management and a symphony. They explain that initially, their data exploration was clear and straightforward, like a solo violinist. But, as Intuitive Surgical expanded without coordination, the data management became chaotic. They decided to use dbt as a conductor to bring harmony into their data management process—something many companies relate to as they scale their operations.

"Data exploration was like the solo violinist: clear and simple. As new members joined, and as the company needs grew…data management quickly became chaotic," says Chris. Their team turned to dbt to resolve this lack of harmony. Chris explains, "Today, using dbt as our conductor, we are becoming harmonious once again."

The importance of thorough qualification in a heavily regulated industry

Brice and Chris emphasize the importance of qualification in their data management process. Given that their work involves life-critical surgeries, they have to rigorously vet their data before it can inform their decision-makers and reach their customers. They explain that it's not just about testing, but also about making sure that the data maintains medical device regulatory standards while remaining agile.

"We thoroughly validate the logic that we use in our data modeling," said Brice. Chris adds, "Given the life-critical nature of surgeries, it's paramount that our data is rigorously vetted before it can inform our decision makers and reach our customers."

The impact of dbt in their data management journey

Brice and Chris highlight how dbt has played a crucial role in their data management journey. "dbt's modular structure and built-in documentation allows developers to find and focus on exactly what pieces need to be changed,” says Chris. This modular structure allows developers to pinpoint changes and ensure that they maintain scrutiny without compromising their agility.

Chris also notes that they’ll continue to use dbt to conduct their data management, stating, "Our ongoing efforts aim to onboard more internal soloists and migrate the tools that they developed over the two decades of [Intuitive Surgical’s] history."

Chris and Brice’s key insights

• Intuitive Surgical uses dbt as a conductor to manage its data, which had grown chaotic as the company expanded
• The company operates in a highly regulated environment, which necessitates rigorous vetting of data before it can inform decision-makers and reach customers
• Intuitive Surgical uses a unified, yet independent, strategy for its dbt projects, ensuring that data teams maintain medical device regulatory standards while remaining agile
• The company has developed internal dbt packages and is considering open-sourcing some of them