Shawn Rosemarin is the Global Vice President, R&D, Customer Engineering at Pure Storage.
Since publishing the first part of this series, an unprecedented critical hardware supply shortage has complicated the landscape. While the component supply crunch remains the headline, it also underscores that AI infrastructure architectures—like the new AI hardware supply chain—need to adapt.
In this article, I’ll explore the principles and models that are pushing conventional, siloed and static storage toward obsolescence and how data can be distributed and managed autonomously.
The Operating Model Has Gone Hybrid—By Design
Gartner projects that “90% of organizations will adopt a hybrid cloud approach through 2027,” meaning data and workloads will span multiple environments by design. This one datapoint nullifies the argument that any single storage array, file system or cloud is “the answer” to a modern AI infrastructure.
The old infrastructure model assumed a primary data center with on-premises applications and a disaster recovery site. The new world has applications and data distributed everywhere. Specifically, workloads now run in your own facility (for security, latency or cost), in hyperscaler clouds (for scale and elasticity) or in “colos” or edge clusters (for compliance).
Data gravity often pulls applications and data closer together, which elevates the importance of data platform decisions. In practice, many organizations are moving toward architectures that blend traditional on-premises environments with private and public cloud platforms, creating a more unified operating model.
For storage, this is driving the need for a seamless and ubiquitous data plane, whether the data is in your facility or a hosted cloud. The same goes for the control plane, where the mechanics of administration—policy-driven tiering, caching, security and replication—can be done consistently across operating environments.
This concept of “seamless ubiquity” has already been proven in the cloud operating model: Over one million data lakes run on AWS S3, and its ubiquity as a standard data plane is precisely why it works—not because of magical hardware, but because it is a consistent, policy-driven abstraction that allows it to run anywhere.
Why Governance Is Now Table Stakes
The regulatory bar has also shifted from “nice to have” and “best efforts” to “mandatory for competitive viability.” The EU Artificial Intelligence Act, now in force, requires high-risk AI systems to be built on datasets that meet defined quality criteria and are governed by documented data management practices. Providers must maintain technical documentation covering data sources, preparation processes, assumptions and bias mitigation—and be able to demonstrate compliance to regulators.
The old model of humans writing wikis, filing tickets and hoping everyone remembered the rules cannot operate at machine speeds. If your data platform can’t auto-capture lineage, classify sensitive fields and enforce retention policies as a matter of course, you’re not just carrying technical debt. You’re carrying regulatory exposure.
From Siloed Boxes To Data Platforms
For decades, enterprises have deployed bespoke storage solutions to handle the needs of specific workloads, leading to inefficient silos across a diverse set of protocols:
• Block storage for databases, using LUNs and volumes optimized for fast random I/O
• File storage for collaboration, shared access and log management across large user bases
• Object storage for unstructured data and flat-scale datasets that support modern AI and developer workflows
This proliferation of systems—and the management layers that come with them—has increased operational complexity for storage and reliability teams.
As mainstream AI drives exponential growth in data sets, more customers are beginning to look “beyond the box.”
The Data Platform Is The Foundation
Storage can no longer be a set of boxes you hope are fast enough and big enough. In the AI era, storage infrastructure, data set management and data intelligence create the platform—the operating model for your data estate. Every dollar saved in data movement is a dollar earned in efficiency. Every policy automated is a compliance risk eliminated. The question is no longer whether to modernize but how deliberately you choose to do it.
A Practical Framework For Technical And Infrastructure Leaders
Before committing to any platform or vendor, ask yourself a useful question: Where does my current environment actually stand?
Signals To Consider
If your storage teams spend more time managing infrastructure than enabling data access, that’s a signal. If your data scientists are waiting on pipelines instead of running models, that’s a signal. If your compliance team is manually reconstructing data lineage for audits—or can’t reconstruct it at all—that’s a signal. And if infrastructure costs are scaling faster than AI workloads are delivering value, that’s the clearest signal of all.
None of these are edge cases. They’re the norm in organizations still running on the old model.
Trade-Offs Worth Weighing
Modernization is not a binary choice between “rip and replace” and “do nothing.” In between the two extremes, there are trade-offs to weigh:
• Speed Versus Stability: A phased approach reduces migration risk but extends technical debt.
• Consolidation Versus Flexibility: A unified data plane simplifies governance, but it will demand discipline around standards in multicloud environments with multiple vendors.
• Governance Now Versus Governance Later: Bolting on lineage, classification and access controls after the fact is exponentially harder than baking these in from the start.
The Realistic First Step
Start with a data estate audit. Map where your data lives, how it moves, who accesses it and what governs it today—not what your architecture diagrams say, but what’s actually happening in production. From that baseline, you can identify the highest-friction points: technical debt, silos that slow inference, lineage gaps that create compliance issues and manual processes preventing efficiency at scale.
That audit won’t tell you which platform to buy, but it will tell you what you’re actually solving for. And that’s an honest starting point for important modernization decisions.
No one is really ahead of the game, especially when almost all architectures have been built for data to serve expanding applications and compute silos. However, the organizations that move deliberately on this will build AI on a better foundation that holds. Those that wait risk finding themselves solving infrastructure problems at the worst possible time: when the workloads are already in production and the pressure is already on.
Forbes Technology Council is an invitation-only community for world-class CIOs, CTOs and technology executives. Do I qualify?
