How to get more power from your data analytics engine
With the value of business data growing, we need to think about where it is worth investing in getting the most out of it. Maybe, it's not where you would expect.
Companies producing cars, trains, ships, or buildings once dominated the corporate world, with their value typically comprised of cash and inventory. Their assets were primarily tangible.
In 1975, 83% of the value of SAP500 companies lay in their tangible assets. The remaining, 17%, were intangible. This is no longer the case.
In 2015 the ratio reversed, and in 2020 average percentage of tangible assets dropped as low as 10%. Today, software companies powered by data dominate the bench of corporate giants.
FAANG (or MAANA?)—Facebook/Meta, Amazon, Apple, Netflix, and Google/Alphabet are the best examples. These are massive technology platforms operating across multiple industries, focused on powerful user experiences, capturing vast amounts of data, and using it to drive their exponential growth.
No wonder that data tops the agenda of many company leaders. But extracting the value from data often turns out to be challenging. And it's perhaps a bigger job than we think.
The path towards data-driven business
We’ve seen massive progress in how companies think about their data in the last year or two.
On one side, we’re seeing a technology renaissance. Storing and processing vast amounts of data is becoming a commodity.
Similarly, data teams are growing. Data Engineers, Analytics Engineers, Data Analysts, and Machine Learning Engineers are finding their way into organizations, including leadership teams. Data leaders—Directors, VPs, and Chief Data Officers are climbing the corporate ladder.
That’s all great.
But turning data into a decision-making machine is not as easy as picking a couple of new SaaS tools, hiring a team of skilled data professionals, and appointing a VP of Data. Or is it?
The hardship of the Data Team
To understand the challenge, we need to look at the entire journey of data in the business.
On average, 110 tools[1] across all departments generate vast amounts of data every day. And in fast-growing companies, the tools and processes are constantly evolving.
Thanks to the modern data stack, this data is captured, integrated, and stored, ready for the data team to use in one of the mighty cloud data warehouses.
But look under the hood of today’s companies, and you will quickly see that even highly skilled data teams with the latest tools are struggling. Something is not quite right.
Why is that?
We’ve accepted the ways of working where business teams have to move fast, without much consideration for the data they produce. Teams swiftly execute changes in the processes, products, and tools without understanding the full impact on their data.
This mindset has several unintended side effects:
Data is not well-designed—it’s a by-product of the process/product/tools and often not created with the analytics use cases in mind.
Data changes cause complexity creep—forcing analytics teams to “fix” data in their data models, overcomplicating the overall logic, and creating a maze of complexity.
Data is entirely missing—in the worst case, teams don’t capture information about their business, leaving data teams and the rest of the company in the complete dark about what is going on.
Data breaks—change executed without consideration for the data often causes unintended consequences and latent errors across the data stack.
And more.
The data teams are right in the middle of it—hundreds of changing data inputs from across the business on one side, stakeholders demanding reliable data on the other. And so, a typical data team can be fire-fighting daily—fixing failing dashboards, broken data models, malfunctioning data pipelines, or missing columns or entire tables. And the data end-users are not entirely satisfied.
This leaves me wondering:
Is it realistic to expect the data team to succeed if the rest of the company operates without much consideration for the data they produce? Does this not perpetually leave the data teams one step behind?
Data beyond ‘a data team’
We’ve made the right steps by investing in data engineering & analytics teams and their technology stack, but to get to the next level, we need to look broader:
We need to change the rest of the company and embed the data agenda deeper across all teams producing and using data. The data team can seed a lot of necessary competencies across the business, but it shouldn’t be just them doing the work.
Here are a few practical initiatives we can work on to move the data agenda forward:
Sell the value of data to non-data professionals
Our data teams are already sold on the value of data—it is their craft, after all. But it’s not a given that everyone in the business believes the same. Why would product managers, engineers, operations managers, salespeople, marketers all invest their valuable time and energy into getting their data right? What do they get out of it? We need to create a positive feedback loop where teams get a return for their investment. It will incentivize the correct behavior, as teams would want to invest for themselves and to provide great data to the rest of the company.Embed the data competence across the business
Teams that are sold on the value of data might struggle to act. They might lack data competence. We should embed data professionals across the business to impact its day-to-day and make sure they can influence how vital decisions get made.Uncover company-wide value chains
We need to create better overviews of flows of data across companies. Teams need to see the impact of their data—to see the value they drive and what essential use cases they might break if their data doesn’t work.Define a robust ownership model
The “the data team owns the data” model is insufficient. All data producers should own the data they create and its quality. It will ensure that the company analytics ecosystem has great clean inputs, maximizing data value for decision-makers. Data has to become a shared responsibility to succeed.
This is not an exhaustive list, but all points have one thing in common: They make data a much more collaborative game that goes far beyond data teams alone.
I expect a lot of evolution on several fronts—cultural, organizational, and technological. Hopefully, we will see data roles more embedded across the business, technology solutions that help teams own and collaborate on their data, more emphasis on data education, and improved incentives.
2021 was a fantastic year for data, and I am keen to see what 2022 brings. Perhaps we will see the data agenda expanding well beyond what is described today as ‘a data team’ and instead involving the rest of the business.
Exciting times ahead.
[1] https://www.statista.com/statistics/1233538/average-number-saas-apps-yearly/
Image by https://unsplash.com/@sigmund