Breaking out of the “data as a service” trap with reusable data products
How a $3.5B startup moved from a reactive mode to a proactive mode to break out of the "data as a service" trap
Many data teams are not set up for success and remain stuck answering ad-hoc questions or service requests from across their organization. With so much time spent responding to the needs of their colleagues, it’s little wonder that data teams struggle to become proactive enablers of their business colleagues.
Chargebee’s Data Engineering team was receiving 350 requests per quarter, 80 of which were repetitive. 70% of those requests were for raw data.
They were falling into a “data as a service trap”, and their data team was spending a significant amount of their time servicing a large volume of data requests and collecting requirements, rather than proactively creating data products that their colleagues could drive value with.
This week, we’re excited to spotlight Chargebee’s journey, revealing how they broke out of the “data as a service” trap by building reusable data products available through a self-service model. You can read the complete case study here.
✨ Spotlight: How a Chargebee broke out of the “Data as a Service” trap with reusable data products
Enabling revenue management on a global scale demands careful attention, a sophisticated architecture, and ocean of data. Chargebee supports more than 100 currencies across 53 countries, integrates with 55 revenue technologies, and maintains more than 30 payment gateways.
In early 2021, Chargebee’s growth accelerated significantly, with a commensurate increase in requests for data. Chargebee’s Data Engineering team was responsible for processing these requests, both from internal colleagues and customers.
“Internal data requests were pushed to the back of the queues as customers were always a priority. This meant that we were not meeting SLAs and there was unhappiness among our stakeholders and lots of escalation, which led to unhappiness within the team as well.” – Lloyd Lamington, Business Solutions Manager.
To meet this increasing volume of requests, the Chargebee team first turned to hire new colleagues but found that the transactional nature of their Data Engineering function made hiring difficult.
Chargebee then turned to automation and standardization, creating dashboards and workflows to respond to repetitive requests. While helpful, these requests were often too bespoke to service with a single view of data. Struggling to meet their SLAs, and with growing escalations to subject matter experts, Chargebee had to find a new way to meet their colleagues’ and customers’ expectations.
They began an evaluation of the Data & Analytics software market, beginning with customer data platforms like Segment, and exploring the capabilities of existing tools like BigQuery.
Focusing on self-service as a potential solution, the team discovered the Active Metadata Management and third-gen data catalogue market. After choosing Atlan, the Chargebee team got to work researching the nature of data requests to ensure they would yield value from the platform as soon as possible.
The priority for Chargebee’s data team was to reduce the volume of requests, especially basic questions related to the location of data.
Over 20 data sources are consumed at Chargebee, including Salesforce, Hubspot, Gainsight, SAP, and Splunk, which are transformed and loaded through Fivetran into BigQuery by their data engineering team. Downstream, visualization and analytics teams consume this data in Tableau and Google Data Studio for reporting and analysis.
Navigating this data estate, system by system, was an impossible task for most of Chargebee’s data consumers. But Chargebee built a solution resulting in a 90% reduction in data request resolution time.
Read how data leaders at Chargebee achieved this with the help of the right tools, processes, and cultural changes in this blog.
📚 From Our Reading List
Doing Data The Hard Way Part 1: Extracting Data by Pedram Navid
DoorDash Identifies Five Big Areas for Using Generative AI by Alok Gupta
Europe Data Salary Benchmark 2023 by Mikkel Dengsøe
LinkedIn: Our Learnings from the Early Days of Generative AI by Zheng Li
Hello Product Data Team, Goodbye Ad-Hoc Work by Sven Balnojan
🗓 Reminder: Tomorrow at the Great Data Debate
In the next edition of Atlan’s Great Data Debate, we have some of the most forward-looking thinkers coming together to debate and discuss the AI-led data stack and the life of a Chief Data Officer in an AI-powered world.
Join us tomorrow by signing up here →
P.S. Liked reading this edition of the newsletter? I would love it if you could take a moment and share it with your friends on social! If someone shared this with you, subscribe to upcoming issues here.