Itâs a problem any student can relate to. Youâre learning a new concept, and youâre stuck on how best to answer a question, or even understand altogether. Itâs the universality of this experience thatâs made Brainly the worldâs most popular education app, serving students, parents, and educators, helping them answer questions and strengthen skills across subjects like English, Math, Science, and History.
Serving hundreds of millions of users, across diverse personas and subject matter, Brainlyâs business generates an astounding amount of data, and their organization depends on it to better serve their users and continue their growth.
In this weekâs newsletter, we spotlight how Brainly transformed its data and analytics strategy. You can read the full story here â
⨠Spotlight: Brainlyâs Journey to Data Mesh
Illustrating some of whatâs worked well for Brainlyâs data team, Kasia Bodzioch-Marczewska, their Domain Lead of Data Engineering, joined Atlan at the 2023 Gartner Data & Analytics Summit in London, sharing the progress her team has made on Data Mesh, how Active Metadata Management can help drive necessary cultural and technical shift, and key takeaways for data leaders considering a similar journey.
Decision on Data Mesh
Recognizing the potential of activating Brainlyâs data, their team began to consider implementing Data Mesh to better organize work, and encourage full ownership and stewardship of their data.
A data mesh is a technical and cultural approach to building a decentralized architecture that organizes data by a specific business domain, providing more ownership to data producers.
The Brainly team began their Data Mesh journey by defining two critical dimensions to master. First was technical, investing in technology that would better enable transparency, team-to-team collaboration, and data quality standards. Second, and perhaps more importantly, was cultural, enabling ownership and accountability, despite a decentralized governance model.
Further enhancing the value they could yield from Data Mesh, and treating data as a product, Kasia and her team found important alignment with the way Brainlyâs product teams were organized and run.
âA few years back we decided, as a company, to decentralize our product teams,â Kasia shared. âEvery product team at Brainly is independent and has their own tools and data.âÂ
While this model paid dividends for innovation and agility, making Brainly the leader in education technology that it is today, the siloed nature and ownership of this data meant frustrating back-and-forth whenever one team needed anotherâs data.Â
âIf we think about this data and this setup, if a program department like Tutoring, for example, wants to utilize financial data, they would have to go and ask. There was a very long process to get access to the right data, and to figure out which data you could use for your analysis,â Kasia explained.
The Data Mesh concept, combined with Brainlyâs unique model of product domain ownership, was a clear and exciting opportunity. But to eliminate the team-to-team collaboration friction inherent to this strategy, Kasiaâs team evaluated the Active Metadata Management market in search of a solution.
Data Catalog â Enhance Data Mesh
âWe figured out that we needed a data catalog to support us with the cultural piece of Data Mesh. We went through several proofs of concept, and through those, we chose Atlan as our data catalog.â
In the early stages of their Data Mesh journey, Atlan has proved to be a crucial partner as Brainly migrates to a new data platform that will better support their new way of working.
âWeâve onboarded into Atlan all of our data sets from both our legacy platform and our new data platformâ Kasia explained. âAnd for migration purposes, weâre using Atlan to identify these sources of the products that are to be migrated to the new data platform, as well as to figure out the downstream objects affected by decommissioning in the legacy platform.â
Brainlyâs new data platform consumes a vast array of data sources, both structured and unstructured, passing through to a raw data lake in AWS S3, Spark and Glue for processing, through to a data mart using Snowflake, and Tableau or Metabase for visualization and analysis.
As users migrate from legacy-to-modern, Atlan serves as their gateway to understanding and applying data and the new platformâs capabilities. âObviously, we have users consuming all of this. We have all of our assets in Atlan; everything integrated into Atlan. Basically, Atlan is the place where all data sharing happens,â Kasia explained.
Ten domain teams at Brainly are already using their new data platform, and increasingly depend on Atlan for crucial context about available data. âWe see more and more people defaulting to the data catalog instead of going through a lengthy process of going back-and-forth (with questions),â Kasia explained.
Read the lessons learned by Kasia the data team at Brainly in this journey.
đ From Our Reading List
Data Scientist: the most ambiguous role of the 21st century by Remi Ounadjela
The future of data by Pedram Navid
A question is a start by David Jayatillake
Palantir Foundry: the data operating system that is not talked about enough by Robert Kossendey
đĽ The Big Reveal of Atlan AI â 7 Days to Go!
A few weeks ago, we announced the launch of Atlan AI as the first-ever copilot for data teams. And itâs almost time for its big reveal. The launch was also covered in Bennâs newsletter.
If you are interested in learning how AI has completely revolutionized the way data teams discover, understand, and work with data, come join the waitlist to explore Atlan AI and save your spot for the launch event next week here.
P.S. Liked reading this edition of the newsletter? I would love it if you could take a moment and share it with your friends on social! If someone shared this with you, subscribe to upcoming issues here.