In this weekly newsletter, I write about my experience as a data leader, lessons and best practices for building strong data culture, and my musing on metadata and the modern data stack. ✨
As a young 21-year-old, I was inspired by the idea of using data science to tackle the big problems facing humanity. At one point, our data team was processing data for 500 million Indian citizens, and billions of pixels of satellite imagery.
Who can explain what this data means?
There are two tables with the same name?! Which one should I use?
Why are our MRR numbers different in the finance and sales reports?
But questions like these were common on our Slack channel. Like any other data team, chaos was an everyday norm for us.
Trust issues were breaking our team apart. The Analyst Teams blamed our Data Engineering Team. Our Data Engineering Team had far more work than they could handle and was frustrated by constant firefighting. They blamed our Consulting Teams for setting unreasonable expectations.
We started building internal tools to navigate through this chaos and help our data team collaborate better. These tools helped our team become 6X more agile. We created global data platforms for the SDGs, built India’s entire national data platform, among other things.
These learnings and experiences led our team to launch Atlan as a unified data workspace for modern data teams. Here’s more on this story.
When diverse people — engineers, analysts, economists, consultants, and scientists — can come together and collaborate effectively, amazing things can happen.
This newsletter is an attempt to share more from this diverse data world, its challenges, data team culture, metadata, memes, and more.
Why subscribe to Metadata Weekly?
Subscribe to get your weekly dose of readings, resources, podcasts, memes, and some food for thoughts on the modern data stack. You also get the complete archive of the Metadata Weekly newsletter. 😻
You won’t have to worry about missing anything. Every new edition of the newsletter goes directly to your inbox. 📨