Three Challenges For The Data Supply Chain

The failure of physical supply chains can have significant and far-reaching consequences. But a faulty data supply chain can also harm a company. Pure Storage outlines three challenges for the data supply chain.

Functioning supply chains are a must for today’s global economy. But in addition to the conventional supply chain, another one is becoming increasingly important: the data supply chain. A data supply chain encompasses the steps in turning raw data into actionable insights so companies can innovate, generate revenue, serve users, and make informed decisions. Like a real-world supply chain, a data supply chain is made up of many different steps and Participants who work together to bring a specific product to a specific place or a specific group of people, even if, in this case, it is actionable or analyzed data.

The typical steps in a supply chain are the creation of raw data, the transformation and integration of that raw data into various systems, and the use or analysis of that data to make it worthwhile for a business. In a data supply chain, the company first extracts data in various formats from disparate and often sealed sources and prepares that data for loading into a repository that can later be accessed for analysis. The data undergoes different processes, such as formatting, enrichment, and cleansing, before it can be analyzed and turned into business value.

This is a straightforward overview of a complex, time-consuming process requiring many tools and actors to execute – from advanced technologies like machine learning and artificial intelligence (AI) to human data experts like data architects and scientists. Exactly how, where, and by whom data is “transformed” depends on what type of information it is and what the organization wants to do with it—from fraud detection to predicting asset failures.

Challenges For The Data Supply Chain

Just as the global supply chain is responsible for getting products into stores and to people’s doorsteps, the modern supply chain that delivers data products faces high cost and complexity challenges.

Complexity

Like technology stacks, data supply chains have only gotten more complex over the past five years. Complexity is never good for a system. However, even in the world of technology, problems are often solved by adding more layers or components as patches that end up slowing everything down and possibly even destroying the system that those patches were meant to fix.

Data supply chains are no different. As with the hard commodities of the global supply chain, the more data the world produces, the harder it is to get that data down the supply chain to create the data products that companies rely on for their data-driven decision-making. Companies tend to manage this flow of data by adding more and more layers to their supply chain management system, which sometimes works initially, but later often leads to costly problems in the form of system failures, downtime and potentially even data breaches.

Costs

Cost is another primary concern of data supply chains directly related to complexity. Gaps and inefficiencies created by complexity throughout the supply chain prevent organizations from fully, timely, and cost-effectively meeting their data analytics goals. The more complex their supply chain, the more expertise they need to keep it running smoothly, and the more time and money they will have to troubleshoot.

Dealing With Structured And Unstructured Data

The significant increase in the amount of unstructured data produced also disrupts the data supply chain. Unstructured data is inherently more challenging to define and process because numbers cannot represent it. The typical way to deal with unstructured data is to dump it in a data lake, but data lakes come with data quality, reliability, and hacking issues.

Companies are trying to solve the problem with money by investing in more appliances and technologies like AI. However, this becomes a vicious circle as they only add more infrastructure to their supply chain that does not scale, exacerbating the cost and complexity mentioned above.