My (almost) musical career and RMIT’s journey adopting dbt from Coalesce 2023

"We want a culture where people want to stay. That's great. That's the idea. But...people may leave, and we do want to be able to keep going when that happens."

- Sarah Taylor, Lead Data Engineer at RMIT University

Colleagues from RMIT University, Darren Ware, Senior Data Engineer, and Sarah Taylor, Lead Data Engineer, come together to draw parallels between music and data. They discuss their experiences in both fields and how they apply lessons from one to the other.

Implementing standards early on can help improve team efficiency and reduce tension

Darren emphasizes the importance of setting up standards early on in a dbt project, noting that it helped him navigate the work of other data engineers when he joined the RMIT team. This approach not only improved efficiency but also reduced tension within the team.

"Set up standards as early as possible. If you can, make it so that it is the computer that's telling people that they can't do it, rather than individuals," Darren recommends. He suggests that having clear standards in place can make coding more navigable and tidy, leading to better team performance.

Sarah echoes his sentiments, noting the value of setting up standards for folder structure and code review. She stresses the importance of shared expectations in keeping the team on the same page and preventing any surprises down the line.

Code review should be seen as a two-way street for learning and teaching

Darren also highlights the importance of fostering a positive environment for code review. He suggests viewing it as a two-way street where both the reviewer and the reviewee can learn from each other.

"I try and encourage people who don't do a lot of code review to go in and do it…As frustrating as it is, taking up quite a lot of time, it is a worthwhile occupation," Darren says.

Acknowledging the good work done by colleagues and using the process as a teaching moment can help create a more positive team environment.

Building for the future involves leaving behind something that holds up well

Sarah mentions that while they are aware that the current platform and methods will eventually become outdated, they aim to create something that their future selves, or other future individuals, can understand and find valuable.

"We want to leave something behind that our future selves, or some other future person, will read and go, 'I mostly understand what they're doing. This holds up well,'” Sarah says.

She emphasizes that it's not just about the technology being old or new, but about the principles that apply regardless of the platform. This perspective can help teams build for the future while maintaining the value of their current work.

Sarah and Darren’s key insights

• It's important to plan for the worst and hope for the best. This applies both in music and data
• Best practices are something to be pursued consistently
• Knowledge sharing is important for maximizing team performance
• It's important to maintain standards and consistency in data work
• Understanding the bigger picture is crucial
• Fostering a positive environment is important for team culture