Are data teams actually worth it? With Barr Moses, Benn Stancil, and Douglas Laney

Part two: snippets from Atlan’s first Great Data Debate of 2023

A few weeks ago, I joined Atlan’s first Great Data Debate of 2023 for an amazing chat with some of my favorite data folks about where the data world is going this year.

In the last Metadata Weekly, I gave highlights from the first part of the debate — all about the hottest topics in the modern data stack, featuring Bob Muglia and Tristan Handy.

This week, I wanted to touch on highlights from the second part of the discussion, which featured Barr Moses (Co-Founder and CEO, Monte Carlo), Benn Stancil (Co-Founder, Mode), and Douglas Laney (Data & Analytics Strategy, West Monroe) talking about the future and value of data teams. This is such a hot topic nowadays, so the debate quickly got spicy and insightful.

Keep reading for some of my favorite snippets about measuring data teams’ ROI and impact, or check out the full discussion here.

---

We just published The Future of the Modern Data Stack in 2023. Check out the insights that Towards Data Science featured in their top 3 reads of last year. Download the report here.

---

✨ Spotlight: Snippets on the ROI of data teams from Barr Moses, Benn Stancil, and Douglas Laney

Are data teams actually worth the money?

The TL;DR version

This year has been all about measuring data teams — assessing the numerator and denominator (the value and cost) of the data function, as Douglas put it. In the end, does that number turn out positive, or are data teams not actually worth the time and money in difficult times?

“It's money time for data teams now. They actually have to prove… all the investment that we’ve made.” —Barr Moses

“It feels like we spend a lot of time defending what we're doing. And at some point you have to wonder, are we defending a bit of a lemon?” —Benn Stancil

“Most organizations are not in a very good position at all, or have been, to value their data function.” —Douglas Laney

Barr Moses:

The big change that happened in the last few years — data teams became way more important in organizations. I think roles became way more clear. There's a lot more investment in data teams. Even in this market, we still see data teams hiring a ton, investing a ton, and leveraging data to actually improve decision-making.

It's money time for data teams now. They actually have to prove that all the investment that we've made is now the time for us to actually use data. And I think, in order to do that, we have to get closer to the business.

I think one of the things that I'm seeing a lot is this breaking point or friction between data teams and business. This has to do with proving ROI — what are do we doing day to day that actually impacts the business using data?

Benn Stancil:

I would disagree that data teams have become really important in the last few years. We've spent a lot of money on them. I don't know if they're important yet.

I think that we have convinced ourselves that we do a whole bunch of important stuff, and we've told all these grand stories about how data teams are really important and companies can't survive without data and all that sort of stuff.

Is that true? Again, it's a narrative that we have. I actually don't know that that's actually born out. I think that that what happens now is it becomes like, all right, we have to actually prove that. Some companies, I think, will. Some won't.

We have to find a new way and it's not going to be exactly right for everybody like this. There's not a “one size fits all” for how data teams work. I think there will be some companies that are like, “Actually, this just isn't worth the time for us. This isn't the thing that is worth some big investment.”

Douglas Laney:

Most companies have an entire department dedicated to procuring office supplies, but not a single person dedicated to procuring data supplies, particularly external data supplies. If we're talking about measuring ROI and the value of data-related initiatives, then we probably ought to have someone with some kind of economic bent focusing on data and analytics.

Right now, due to antiquated and arcane accounting practices, data is not considered a balance sheet asset, even though it clearly meets the criteria of one. Only when the accountants come around to that will the rest of the world be compelled to treat data as an asset. And then we as data professionals will be more revered in our organizations.

Benn Stancil:

It feels like there’s a lot of circular justification for what we do. There's a lot of “how do we measure the ROI?” Well, you can't really do it. We do it in these sort of soft ways. You go to a sales VP and say, “Hey, why are you here?” They're like, “I can answer that in 10 seconds, no problem.”

It feels like we spend a lot of time defending what we're doing. And at some point you have to wonder, are we defending a bit of a lemon?

Key metrics to measure the value of data teams today

The TL;DR version

With this year’s economic uncertainty, showing ROI is more important than ever before. If you were running a data team today, what metrics would you use to measure and communicate your value?

“Usage metrics. Nothing's perfect, but I would start with that.” —Barr Moses

“If the marketing team had to take from their budget to say ‘We want to fund data work’, will they actually do it? When push comes to shove, is it worth money to those folks?” —Benn Stancil

“We need to get away from thinking about people using data.” —Douglas Laney

Benn Stancil:

My answer would be NPS — the quantified reaction on people's faces when you tell them they no longer have someone on the data team supporting them, to see how upset they are.

Then I think this is a little bit of a political number — adoption of the things that you're building. I don't necessarily know that that means you're delivering value, but if you are advocating for your data team, you can often create numbers that show a good bit of adoption. People relying on these dashboards — do they rely on these dashboards to make good decisions?

Douglas Laney:

If I had to pick [two], one would be connecting data quality to business process performance — whatever data quality or aggregate of data quality metrics you want. Some aggregate of accuracy, completeness, integrity, timeliness, whatever. And then connecting those to the KPIs for the business processes where that data is used.

The second would be looking at the denominator side of data's value proposition. What is the cost to manage, store, secure, collect data, generate data? And then the numerator side — for any given data asset, what is its contribution to a revenue stream?

Barr Moses:

I would say, the incremental dollar value that your data team is driving. I think that should be something that teams aspire to, if they can't do it already. And as a leading indicator to that, usage metrics, to the extent possible. Nothing's perfect, but I would start with that.

Then maturity metrics like what Doug mentioned around security, scalability. I'm obviously biased, but reliability and trustworth[iness] of the data, I think, are super important. It's just really hard to use the data if you can't actually trust it.

Douglas Laney:

I'm not really keen on usage. You're probably talking about that as some sort of proxy for benefits, right?

Barr Moses:

That's right. If you're rolling out a BI solution and there's only one person using it, is that a lot or a little? I don't know.

Douglas Laney:

But I mean, if that one person is using it to maintain minimum inventory levels that are 20% below historical trends, that could be really valuable.

Barr Moses:

Totally agree. That's why I think tying to dollar value is the most important.

Douglas Laney:

I also think we need to get away from thinking about people using data. Increasingly, it is applications and systems themselves that are the consumers of data. So we need to get out of that box of thinking about people using data, because that's just not going to be the case going forward.

I try to avoid tying it to people. Tie it to process value, not the number of people using it.

---

---

📚 More from my reading list

• 10 lessons learned in 10 years of data: part 1 and part 2 by Mehdi Ouazza

• Where is data science headed in 2023? by Towards Data Science Editors

• How misused terminology is damaging the data field by Ivan Reznikov

• Optimizing data modeling for the data-first stack by Animesh Kumar

• Analytics engineer- a glorified BI engineer? by Madison Schott

• There’s dashboards all around us that are USED by Randy Au

• Data governance at Brainly by Katarzyna Bodzioch-Marczewska

• Data mishaps night organized by Caitlin Hudon and Laura Ellis

P.S. Liked reading this edition of the newsletter? I would love it if you could take a moment and share it with your friends on social! If someone shared this with you, subscribe to upcoming issues here.