Secret Escapes is a UK-based luxury travel company that specializes in curated trips to locations across the world. The company acts as an agent for hotels and operators and markets deals to its members.
As a digital travel company, Secret Escapes generates a huge amount of data. This data is used to build key insights—such as how users interact with the company website, or the status of relationships with hotels and operators—and improve core business areas and customers’ experience.
Foundations of Secret Escapes’ Modern Data Stack
In 2018, Secret Escapes invested in modernizing its data stack to ensure resilience through trade volatility.
“When I first joined, we only had Snowflake and a foundational infrastructure that allowed our pipeline to ingest data into our warehouse,” said Robin Patel, Secret Escapes’ Head of Data & Analytics Engineering.
Over the next few years, the Data Platform team worked to build out the business’ data warehouse, with a vision of easier, more efficient reporting.
Forever an evolving product, the data warehouse soon proved to the business that it was capable of serving data for multiple use cases—whether analytics or feeding external systems. However, its initial success led to more data requests, and a new problem emerged; the Data Platform team became a bottleneck for creating new datasets. Secret Escapes needed a solution that would empower analysts to provide data products, not just data engineers within the Data Platform team.
The team demoed and ultimately chose dbt Cloud with this goal in mind: providing autonomy to analysts around the business, while also instilling data governance and development principles.
Democratizing and de-risking legacy data systems using dbt Cloud
One of the first datasets the team set their sights on improving using dbt was an existing system for modeling and comparing the return on investment (RoI) of Secret Escapes’ marketing initiatives. Before introducing dbt Cloud, the model was functional but cumbersome.
5 years ago, Secret Escapes commissioned an agency to create their custom marketing RoI solution; the system connected to external marketing systems to ingest data into an old MySQL database and then into an Access database where costs were married with revenue by ingesting a CSV mapping file.
“It was not only complex but also isolated and risky,” explained Robin.
The team knew there were improvements to be made, and took on the project as their dbt POV and evaluation:
“Rather than incurring migration costs from external contractors to build more custom connectors in our warehouse, getting our engineers to model the data, and then working with business stakeholders to understand if a model is fit for purpose, we instead taught our analysts how to use dbt Cloud, gave them the data, and they did the rest,” said Robin.
Before dbt, similar requests would have sat in a queue for months.
“It wouldn't have met prioritization, so it would have taken a couple of months for us to pick it up—and then any iteration needed would require a new ticket, which takes time to implement. With dbt, analysts got the data product delivered quickly, and the marketing analysts own all the logic,” explained Gianni Raftis, Head of Data Products & Strategy at Secret Escapes.
Discovering dbt Cloud: More than Documentation
Gianni and the team first heard of dbt in peer discussions on documentation.
“A lot of data professionals are mentioning dbt, and I’d heard great things about its documentation features,” said Gianni.
Secret Escapes runs a monthly hackday: “On the last two days of each month, engineers can work on whatever they want, free of business requests,” explained Gianni. “We started to experiment with dbt and realized that it could do so much for us. And it just snowballed from there.”
Data Platform then tested dbt in their hack day and they soon realized that the solution could provide much more than they expected.
Since implementing dbt Cloud, the data team at Secret Escapes has found that they’re able to give autonomy to users outside of the Data Platform team to model and structure their data and develop pipelines.
Uniting Data across Subsidiaries
One of the most significant changes brought about by using dbt Cloud is the integration of data across subsidiaries.
“Our business incorporates a few subsidiary brands, acquired over the years,” explained Robin. “From a group-level perspective, we've got several technology stacks with different ways of reporting metrics. Because they all have slightly different products, it's not as simple as just rolling them up together.”
Using dbt, one single analyst was able to take the different data sets and combine them together for group-level reporting—removing significant amounts of manual labor compared to their previous process, which was heavily reliant on Excel and not automated.
"dbt has empowered me to build a unified data model that incorporates all trading metrics for each entity with Secret Escapes, a model previously nonexistent in the business. This has streamlined our reporting processes, enabling the automated creation and distribution of daily and monthly KPI reports to the business and board of directors," said Dharmita Bhanderi, Senior FPA Analyst.
Increased C-Suite Understanding: Business Predictability
Another area where dbt Cloud proved itself invaluable was in communicating information to stakeholders.
“We had a high-level need to better understand our user base,” explained Robin. “Using dbt Cloud, we were able to very quickly build something that would have previously taken an exorbitant amount of time.”
The solution’s utility, combined with the team’s ability to iterate further leveraging dbt, helped sell it to other departments at Secret Escapes.
“The model delivered great value, helping the wider business and the C-suite understand a lot more about our user base—with metrics such as how often a user drops from our service flow and becomes inactive,” Robin added. “Now we can use our model to better predict how our business will perform based on how much we spend.”
Rapid Iteration: Modeling Email A/B Tests
dbt’s use cases at Secret Escapes extend beyond reporting. Building on top of dbt’s models, the team implemented a series of A/B tests on their email and web recommendations to test different approaches and strategies and judge their performance.
“We can maintain and quickly adjust code in dbt Cloud—without restarting the whole development process within our warehouse,” said Robin.
Because Secret Escapes can track the performances of the A/B tests within dbt Cloud, it’s easier and quicker to output results and iterate.
“I managed to build a whole model module around A/B testing with loads of different data sets and create a scalable product,” Gianni added. “Now, every test I run is all in dbt—and it runs every day. Even the CRM (customer relationship management) executives are using the model, which is just brilliant.”
Growing Analyst Confidence
With dbt Cloud’s assistance, the Secret Escapes data platform team is rapidly developing its capabilities. Robin is careful to dedicate time and resources to the people using the data tools as well to ensure the technology meets users where they are.
“We can't expect our analysts—not to mention everyone else in the business that wants to use the data—to be advancing at the pace of our technology. It's just not sustainable. Instead, we try to empower our end users.”
Rather than serve raw data, the Secret Escapes data team provides other teams access to a semantic layer of approved core data products that have already undergone modeling & transformation.
“If we had just given everyone the full raw datasets to play with, most individuals would feel overwhelmed,” explained Robin. “They would be hesitant to leverage our centralized metrics, resulting in redundant data, redundant processes, and disengagement from our products.”
By tailoring which datasets users interact with through dbt Cloud, Secret Escapes maintains team engagement and thoughtful collaboration.
“It stops data being misinterpreted downstream, and gives analysts greater confidence by serving as a sort of safety net.”
Guard Rails: Enabling Risk-free Data Development
Echoing the spirit of safe data usage, Secret Escapes used dbt Cloud to create guard rails that ensure data modeling occurs in a safe, governed manner.
“Some of the guard rails are there to ensure that the analysts’ work never affects any of the core data sets,” Robin explained. “Anything they do rests on top of the core data, appearing in a completely separate database. This keeps things from getting messy, letting the analysts get to work without accidentally disrupting our core data. And because the development layer sits within the same tool, people can still navigate it easily.”
The team has also used dbt Cloud to separate certain data into departmental layers.
“We have sub-sections inside our dbt database and schemas specific to departments,” Robin added. “If I’m looking for CRM data, for instance, I can just go to the CRM schema.”
Far from hindering analysts, these guard rails ensure that dbt can be navigated with confidence.
“The guard rails reduce risk and help analysts boost their knowledge and confidence within dbt,” said Robin.
Looking to the Future: Training and Expansion
Going forward, the data team is hoping to further push the benefits of dbt Cloud out to the wider organization.
“We’re going to be introducing targeted sessions around documentation and testing, as well as more community-oriented initiatives like drop-ins,” said Robin.
Meanwhile, with analysts now safely and confidently managing the data they need, Secret Escapes is looking to make the most of its modernized web analytics capabilities.
“The next steps will be polishing our insights, and helping people unlock dbt’s complementary features like documentation and freshness.”
