Data management from 1000 years ago: How the pyramids were built using data
Ancient logbook, analysis, documentation, management, and story of a great data team behind it all ✨
Welcome to this week's edition of the ✨ Metadata Weekly ✨ newsletter.
Every week I bring you my recommended reads and share my (meta?) thoughts on everything metadata! ✨ If you’re new here, subscribe to the newsletter and get the latest from the world of metadata and the modern data stack.
I’m writing this newsletter while enjoying the great weather and beaches in San Diego, and over the next few weeks, I’m super excited to meet the data community IRL. I’ll be in Las Vegas (for the Snowflake Summit) and San Francisco (for the Databricks summit). If you are around, I’d love to catch up in person… and invite you to join Atlan’s community parties in these cities!
This week, I decided to take a break from writing about metadata and data culture to focus on something that has always fascinated me — how data teams have always been behind humankind’s greatest achievements (even though they weren’t called that at the time).👇
✨ Spotlight: How the Ancient Egyptians Used Data to Build the Great Pyramids
Checklists and spreadsheets have been in existence for thousands of years. The Great Pyramid of Giza (also known as the Pyramid of Khufu) is 481 feet (146.6 meters) or 280 Egyptian royal cubits of breathtaking stonework. It has survived over 4,500 years of wars, desert storms, and other natural calamities.
How did the ancient Egyptians build one of the most resilient infrastructures humans have ever known? Spoiler alert: data had an important role to play.
It’s believed that the Egyptians took 2 decades, between 2560 B.C. and 2540 B.C to finish constructing the pyramid. This means the Egyptians laid at least one block—one Asian elephant—every five minutes.
According to archaeologists, it took roughly 4,000 primary laborers (quarry workers and masons), supported by 16,000–20,000 secondary workers (ramp builders, tool-makers, and suppliers of food, clothing, and fuel) to build the massive monument.
The Masters of Organizing Data 👑
Archaeologists have found ancient papyri that stand testament to the mega-organizational skills of the ancient Egyptians.
One was a logbook recording the everyday activities for a team of 40 men. Yet another document shows several accounts of food rations and deliveries from various parts of ancient Egypt. All these inscribed papyri are well dated and detailed — one of the earliest examples of spreadsheets and checklists.
The scribes of ancient Egypt also documented their calculations and theories for building all the ancient pyramids on mathematical papyri.
Some of these include:
The calculations for the number of blocks required
The precise angle of the sides with the ground (for Khufu, it’s 52 degrees)
The geometry of the overall structure
The water-to-sand ratio
The calculations to align the Great Pyramid of Giza to true north
Data Science in Action 🤓
Moving heavy blocks on sand generates friction, which creates piles of sand on the path and makes transportation a nightmare.
So how did the ancient Egyptians manage to haul massive stone blocks across the desert? Thanks to the river Nile and brilliant application of science.
In 2560 B.C., the pyramid site in Giza was surrounded by the floodplains of the Nile. The Egyptians opened dikes to divert the river water into man-made canals that led directly to the pyramid. They used these canals to transport huge blocks from the famous limestone quarries of Tura — a town along the Nile — to the city of Giza.
This only worked for the lighter stones (up to 15 tons), so the Egyptians had to haul the heavier stone blocks, such as granite, across the Sahara desert on wooden sleds. They accomplished this Herculean task by figuring out the principles of fluid mechanics.
When sand is wet, the sand grains stick together because of the water and reduce friction. However, pouring too much water will increase friction. The Egyptians determined the ideal amount of water to be poured (2-5% of the sand volume) and reduced pulling force by 50%.
How did they determine the exact ratio? By experimenting and documenting their findings and formulae. Through documented experimentation, the ancient data scientists in Egypt were able to drastically reduce the time and effort required to build the Great Pyramid.
So as it turns out, humans of data were behind one of the best timeless masterpieces known to humankind, doing data collection, analysis, documentation, and management. ✨
Here's a deep dive blog about how the great pyramids were built using data and science 1000 years ago.
I’ve always believed that the most amazing human achievements — past, present, and future — have an amazing data team behind them. From 30-minute pizza delivery and electric cars to someday curing cancer and putting people on Mars. These stories of great data teams inspire me every day. What gives you your daily dose of inspiration?
🌟 Data Jobs for the Data Community
It has been a brutal week with news about mass layoffs & the broader economic downturn affecting teams everywhere. For friends in the data community who're looking for roles, here are some resources that can help:
The moderndatastack.xyz newsletter features open roles in data and analytics in their weekly edition of the newsletter
Companies like Modern Data Stack Recruitment help high-growth companies hire modern data stack professionals, and I'd recommend reaching out to them if you're interested in roles.
Modern Data Jobs (a weekly newsletter by my colleague Surendran) features open roles and companies that are hiring right now.
If you’re looking to move into a software engineering or analytics engineering role, check out the Foundry Program. Huge shoutout to dbt Labs for curating this program! They just announced the details for the next edition. Check it out here.
Do you know any more helpful job boards/ resources for the data community? I’d love to amplify those and share them with the readers of this newsletter.
❄️ Atlan @ the Snowflake Summit
Everybody in the data community is talking about it! 😉 It was hard to imagine being in the same room full of thousands of data people. Thanks to the pandemic. But it’s FINALLY happening next week.
We have a bunch of events happening throughout the Snowflake Summit, and if you’re around, I’d absolutely LOVE to meet you. Here are some you can RSVP for:
👉 Roll the Modern Data Stack slot machine at Atlan’s booth #627 during the summit. There’s some exclusive swag from Atlan, dbt Labs, Fivetran, Hightouch, and Sigma.
👉 Data After Dark: Atlan, Hex, Hightouch, Monte Carlo, Prefect, and Snowplow are all coming together on 14th June to host the best party for the data community – great food, mixed drinks, and good vibes.
👉 WeWork’s Insider Session: On 15th June, I join Harel Shein (Head of Data Engineering, WeWork) to share how WeWork built a culture of trust, transparency, and governance with Atlan and Snowflake.
👉 Hit the Jackpot: Join Atlan’s Happy Hour with Hightouch on 15th June
Lastly, We've saved a few spots for Metadata weekly readers for an intimate, closed-door dinner with Atlan, our partners, and our customers. I'll personally be there and I'd LOVE to catch up in real life! 💙 RSVP here and we'll reach out with invites.
P.S. Liked reading this edition of the newsletter? I would LOVE it if you could take a moment and share it with your friends on social.
See you next week!