Paul Speciale, Chief Technology Evangelist at Scality.
Global data volume and storage needs continue to rise at a rapid rate. By 2030, the volume is expected to surpass 700 zetabytes, according to IDC’s Global DataSphere analysis. Fueled by AI workloads, unstructured data and more edge computing data, the storage market will experience similar robust growth. The global software-defined storage (SDS) platforms and applications market is projected to grow from $28 billion in 2024 to more than $107.7 billion by 2030, according to Research and Markets.
The increase in data flow and storage capacity demand has prompted more enterprises to use autonomous data management to better control workflows and preserve staff time for other high-value tasks. Coupled with AI agents, it’s a strategy that can help create a more workable approach to managing data volume.
The Rise Of Autonomous Data Management
Autonomous data management is not an entirely new concept. Database management systems have long automated tasks such as storing, retrieving and updating data. AI and agentic AI now extend that concept, allowing systems to engage in more complex data-management tasks with human direction and oversight.
The exponential increase of data requires taking AI and agentic AI to a higher, more integrated level of support. Viewed holistically, enterprises can improve overall operations. For example, autonomous AI can help keep customer financial data current, retrievable and compliant while flagging anomalous access requests for human review. Multiply this by terabytes of data flowing through organizations daily, and it’s evident the data cannot be handled by humans alone, nor can it be securely organized, stored and retrieved without the help of AI processing.
Object Autonomy
Within that broader storage mix, object storage can be well-suited to autonomous management of large-scale unstructured data and certain AI workloads. Its rich metadata, API accessibility and policy-based management can support automated data placement, life cycle management, retrieval and governance across distributed environments.
Over the last 15 years, object storage has become a foundational form of cloud storage, especially for large-scale unstructured data such as images and videos. Its scalability also makes it increasingly relevant to emerging AI infrastructure requirements. For example, high-performance, S3-compatible object storage is increasingly being used as a shared data layer for AI infrastructure, providing scalable capacity for model weights, growing context data and other data needed by GPU-intensive workloads.
However, this doesn’t eliminate the need for careful architecture and governance. IT administrators must stay diligent about access controls and ensuring API credentials are properly configured. They also must structure access to certain data by region to meet data sovereignty requirements. Organizations additionally need to determine whether applications can effectively use object APIs and whether object storage provides the latency and performance characteristics required for each workload.
Humans And ROI
Autonomous data management ultimately is not about removing people from storage operations. It is about determining which decisions technology can make safely and repeatedly on its own and which require human judgment, approval or intervention. As AI data volumes grow toward exabyte scale while IT headcount remains constrained, that distinction becomes increasingly important.
Human-in-the-loop principles establish those boundaries. IT administrators and architects define operational policies and security standards. Compliance teams establish requirements for access, encryption, retention and sovereignty. Business leaders determine acceptable risk. Humans also remain responsible for context-dependent decisions such as data classification, access policies, archiving and approving higher-risk system changes.
The most effective autonomous data-management strategies establish clear levels of autonomy rather than simply pursuing more automation. Routine, well-understood processes may operate automatically; higher-risk actions can require approval and exceptions can be escalated for human review. Organizations should also regularly evaluate whether automated policies are producing the intended security, performance, compliance and cost outcomes.
For enterprises seeking a return from AI, the goal should be a powerful human-AI collaboration: Technology handles the scale, speed and repetition of modern data management, while people provide the judgment, governance and accountability that autonomy still requires.
Forbes Technology Council is an invitation-only community for world-class CIOs, CTOs and technology executives. Do I qualify?