Getting value from data | 101 Ways

As organisations have explored Generative AI many have found that poor data management is a blocker to scaling adoption. As the technology matures these issues – from poor quality, inaccessible data to an inability to demonstrate business value – will need to be addressed if an organisation is to reap the promised benefits of AI. 

In the shorter term, there is significant value to be found in using data effectively with today’s tools. Becoming data-driven has been a priority for years, but few organisations have reached a level of maturity where they can honestly claim to have achieved it. There are many reasons for that, both technical and organisational. From a technical perspective data is often hidden, inaccessible, or poor quality. Organisationally, data teams struggle to collaborate effectively with business stakeholders and data initiatives struggle to show business value. 

Finding an effective way to address these challenges is a top priority for data leaders today. One approach increasingly being adopted is to apply Product Management techniques to data to create Data Products.

What is a data product?

A data product is a high-quality, reusable data asset that is accessible to people across the organisation. It should contain all of the data, metadata, policies and pipelines needed to be used or adapted by authorised users. Following product management principles, each product should be owned by a Product Manager with responsibility for creating the product roadmap to ensure value is delivered to users and to guard against scope creep. 

By standardising the approach to using data, enabling reuse of assets, and aligning data initiatives to defined business value, organisations adopting a data products approach can generate significant value from their data while also reducing the costs. A recent Harvard Business Review article found that the time to implement new use cases can be reduced by up to 90%, TCO reduced by 30%, and risk and governance challenges lessened.

The idea of treating data as a product isn’t new but interest in the approach has increased since the concept of the Data Mesh was introduced. One of the key principles of a data mesh is that data should be treated as a product, with self-contained and ready-to-use data artefacts available across the organisation. In a data mesh the data products allow for self-service by users, with self-service platforms allowing access to products that remove the complexity for end users and support lowering costs through re-use of assets already created. Central to this approach is a high-quality and trusted data catalogue through which users can discover and access available products. This data catalogue can be built iteratively by each data product as they process data sources and produce new insights.

The benefits of a data product approach

Adopting a data products approach offers a way to address many of the challenges that have existed since the big data era. Then, the idea was that moving data into a data lake would allow users to find and select the data they needed to answer their questions. In reality, lakes quickly became expensive swamps full of data of varying quality, degrading trust and requiring specialist skill sets to explore. 

The central idea with data products is that they should be created to meet the needs of a specific group of users. Those needs are not solely technical, but include UX and UI, security, trust, accessibility, etc. By first understanding the users and their needs it is possible to put in place metrics to understand whether the investment made is generating a sufficient return to continue. A product roadmap ensures that as those needs change the product can be updated to continue generating value.

Security and governance

Data products can have security controls specific to their intended use. This allows granular access control with only authorised users given access. It also allows for better data minimisation, with only the data required to meet users needs included. 

As part of adopting a data products approach security and governance should be standardised. This includes the use of automation where possible to increase security, governance and auditability.

Trust

Each product should have a product manager, acting as a central point of contact and responsible for ensuring the product meets users needs. Ensuring high-quality, accurate data is delivered to users increases the trust in data and supports adoption to support decision making.

Value

Using data effectively requires users to be able to access the data they need when they need it, and to trust the data they are using. Partly this is about data platforms but it’s also about the operating model. Adopting a data products approach ensures that investment is targeted at the areas where it can deliver the most value. It also keeps those investments aligned with the business strategy, helping organisations to become data driven.

Cost

Data products offer significantly lower data management costs through better reuse. Currently teams will create data assets for a particular purpose i.e. a report, dataset, pipeline etc. Other teams may create similar assets, duplicating effort and increasing costs, and creating confusion over which version to use. By cataloguing the products that have been created and making them available to users who need them much of this re-work can be avoided. It also reduces the need for expensive specialist skills.

Getting started

Adopting a data products approach first requires an understanding of how data is used today and the maturity of current data management practices. This work can help to shape a vision for how the organisation intends to use data and what needs to be done to get there. Ultimately a new operating model for data is usually required that aligns the business, technology, and data strategies and ensures data teams are effectively organised to support them. 

There are a number of technical paradigms that can be used to support data products, from Data Meshes to Data Fabrics to shared data platforms. Which one is most suited will depend on the data needs of the organisation and current maturity levels. In all cases thought needs to be given to defining right-sized governance and security policies to ensure data is used safely. These may be centralised and owned by the data management office or left to domains to ensure adherence to the agreed frameworks. 

Each data product requires a product manager to own the strategy and roadmap and a team to create, enhance, and maintain the product. The team is responsible for ensuring users needs are understood and the product meets these, aligns to the overall strategy, and adheres to any policies that have been agreed. 

It usually makes sense to begin in one area, demonstrate the value of the approach and look to scale from there. Often there are groups that are already using data effectively at a local level and have the required skills to quickly show value with data products. An exemplar team approach can help with improving data literacy. As the team showcases their work more people will grow to understand the value data offers and start to consider how they might use it to support their objectives.

How 101 Ways can help

101 Ways have years of experience in helping clients adopt effective product management practices and in implementing modern data management approaches. Using this experience we help clients to understand the value they could get from data and create a roadmap to help them realise it. We typically begin with a data maturity assessment to understand how data is being used today, the quick wins that could be achieved and the longer-term initiatives required. 

If you’re interested in adopting a data products approach, we would love to help you get started.

References: https://hbr.org/2022/07/a-better-way-to-put-your-data-to-work