Trends driving the new world of metadata, Segment for metadata, and more
✨ Spotlight: 5 Trends Driving the New World of Metadata in 2022
Welcome to this week's edition of the ✨ Metadata Weekly ✨ newsletter.
Every week I bring you my recommended reads and share my (meta?) thoughts on everything metadata!
This edition includes the 5 trends driving the new world of metadata, my views on the service mindset, some picks from my reading list, and events you won’t want to miss out on. Let’s get started! 👇
✨ 5 Trends Driving the New World of Metadata in 2022
Starting around 2016, the modern data stack went mainstream. Today’s modern data stack is easy to set up, pay as you go, and plug and play — people won’t put up with anything else these days!
In an ecosystem of increasingly easy, fast, interconnected data tools, the old idea of metadata — passive, siloed data inventories, powered by an army of data stewards — just doesn’t cut it anymore. Many of the earlier second-generation data catalogs still need significant engineering time for setup, not to mention at least five calls with a sales representative to get a demo.
Last year, we hit some major landmarks in the world of metadata. Gartner scrapped its Magic Quadrant for Metadata Management, companies started asking for third-generation data catalogs, and modern metadata companies (like Atlan ;)) launched and raised some serious VC money. All of this actually prompted me to add metadata as one of my six key data ideas for this year.
Today metadata is itself becoming big data, and technical advances (i.e. elasticity) in compute engines like Snowflake and Redshift make it possible to derive intelligence from metadata in a way that was unimaginable even a few years ago. As metadata increases, and the intelligence we can derive from it increases, so too does the number of use cases that metadata can power.
But even the most data-driven organizations have only scratched the surface of what is possible with metadata.
Metadata is on the cusp of fundamentally changing how our data systems operate. Read about the 5 trends driving the new world of metadata in 2022.
❤️ Fave Links from Last Week
Segment for Metadata by Stephen Bailey
Defrag, the world’s first, only, and best “Data Platform Data Platform”, today announced that it received $42 million in Series A financing in a round led by _tbd Ventures. Tate is in it for the long haul. “No one knows what the data stack will look like in ten years, but I can guarantee you this: metadata will be the glue.”
Okay, I kid you not — I first saw this and my FIRST reaction was, oh, someone wrote an article about Atlan ;) While a further deep dive confirmed that this was just Stephen shit posting, he’s talking about EXACTLY the same concepts that I talked about in my product sketch last year about the “Anatomy of an Active Metadata Platform” — though, I admit, the name “Segment for Metadata” is vastly more appealing. ;)
I firmly believe that the old days of Data Catalog 2.0s where metadata is siloed into another tool are over. In the new world, context should be available, where you need it and when you need it. When a user is in a Looker dashboard and wants to know if they can trust the insights and if the data is updated, they do DON’T WANT TO GO INTO ANOTHER TOOL. They want the context right there.
The metadata plane has the potential to do this — to bring together metadata from siloed sources, make it connected and useful, and push it back into tools in the rest of the ecosystem to vastly improve user experience and reduce chaos.
📘 More from My Reading List
Data Products Can’t Be Black Boxes by Eric Weber
Your First Data Hire by Andrew Bartholomew
Highly Opinionated Integrations by David Jayatillake
Data as a Product vs Data Products by Xavier Gumara Rigol
🗓️ Upcoming Events
💫 Datanova 2022: The Data Mesh Summit by Starburst | 9th-10th February
I’m excited to speak about unlocking the value of data mesh through active metadata at this year’s Datanova Summit by Starburst! Sign up here.
I'll see you next week with more interesting stuff around the modern data stack. Meanwhile, you can subscribe to the newsletter on Substack and connect with me on LinkedIn here.