In 2024, China experienced a complex situation with AI chips: overcapacity in some areas, while simultaneously facing shortages in high-quality compute needed for advanced AI development.
This contradiction is not merely a technical glitch or a byproduct of geopolitical maneuvering—it is a deeply human story of ambition, improvisation, and the unintended consequences of a gold rush mentality.
Let’s start with an image: a vast landscape of idle data centers across China, filled with some of the world’s most advanced GPUs, waiting for a purpose. At the same time, DeepSeek, the AI company making waves with its recent breakthroughs, claims it is compute-constrained—lacking the power necessary to build the next generation of AI models. How can both things be true?
To understand this, we need to look at the recent past. When the United States imposed restrictions on China’s access to cutting-edge AI chips, the response from Chinese companies, local governments, and state-backed telecom giants was swift and predictable: stockpile. They did what human beings have always done when faced with scarcity—they hoarded. They bought up Nvidia’s chips, built AI data centers, and created vast computing clusters in anticipation of future demand. Meanwhile, Chinese buyers continue to circumvent US export controls to order Nvidia’s latest AI chips, including the new Blackwell series, through third parties in nearby regions. But in their rush to prepare for an AI-driven future, many failed to ask a fundamental question: what exactly are we going to do with all this computing power?
The Inefficiency Problem
The first explanation for the paradox is logistical. China has added at least 1 million AI chips in 2024 to its compute capacity. While this is a significant number, it’s important to note that the US is estimated to have several times more AI chips in operation. But China’s chips were not deployed with efficiency in mind. Instead, they were spread across data centers of varying quality, often in locations with little demand. Companies and governments, eager to participate in the AI boom, built infrastructure without a clear strategy, leading to an abundance of what might be called “low-quality compute.”
Imagine a world where millions of people buy expensive concert pianos, believing they will one day learn to play. But instead of placing them in concert halls or conservatories, they scatter them across small, poorly maintained storage units. The pianos exist, but their potential is unrealized. That, in essence, is what happened in China’s AI ecosystem.
The Short-Term vs. Long-Term Demand Problem
The second explanation lies in timing. In 2023, there was a frenzy to develop foundation models—the massive AI systems that underpin everything from chatbots to automated factories. But in 2024, many of these efforts stalled. Some companies gave up, realizing they lacked the resources to compete. Others pivoted to AI applications rather than foundational AI research. As a result, the demand for model training—the most computationally expensive AI task—dropped.
At the same time, demand for inference—the process of running AI models on trained data—began to rise. But inference requires a different type of infrastructure. Training is a marathon that demands massive, centralized clusters of computing power. Inference is more like an intricate dance, with AI models deployed across multiple environments, from smartphones to factory floors. The infrastructure China built in 2023 was designed for training. In 2024, the market shifted, leaving an overabundance of training compute and an undersupply of inference compute.
The “Fake” AI Clusters
Another complicating factor is the phenomenon of “fake” and “pseudo” 10,000-GPU clusters. Some companies bought enough GPUs to theoretically form a large-scale AI computing center but then deployed them in multiple small, disconnected data centers. Without high-speed networking and the right software architecture, these chips could not function as a true, unified system.
This is a classic case of mistaking accumulation for capability. Owning thousands of GPUs does not automatically translate into competitive AI research, just as buying a hundred Ferraris does not make one a world-class racing team. Many of China’s AI clusters exist more as financial assets than as functional research tools.
The Government’s Course Correction
The Chinese government has not been blind to these inefficiencies. In response, it has begun restricting the construction of new data centers unless they meet specific location and infrastructure criteria. It has also encouraged cloud computing, pushing companies to share computing power rather than hoarding private GPU clusters. In theory, these moves should help correct the imbalance by centralizing high-quality computing resources and making them available to AI researchers who actually need them.
The Chinese government has also restricted the construction of new data centers unless they meet specific location and infrastructure criteria, and has encouraged cloud computing to improve resource utilization.
But here’s the real question: does any of this matter in the long run?
The Historical Precedent
Consider the case of America’s railroad boom in the 19th century. In the rush to industrialize, companies built tracks everywhere, often with no regard for actual demand. Some of these railroads became useless, mere relics of speculation. Others, however, found their purpose as industries and cities grew around them. Over time, the initial chaos gave way to a more efficient system.
The same will likely happen with China’s AI infrastructure. The overcapacity of today may be the foundation of tomorrow’s breakthroughs. While many of these idle GPUs are currently going to waste, they represent an investment in a future where AI applications are ubiquitous. The companies that adapt—consolidating computing resources, shifting to inference, and refining their deployment strategies—will emerge stronger. Those that do not will become footnotes in the history of China’s AI rise.
The Takeaway
While DeepSeek’s achievement is significant, it’s important to note that access to advanced chips remains crucial for long-term AI development. As one expert noted, ‘If next-generation models require 100,000 chips for training, export controls will significantly impact Chinese frontier model development’. The AI landscape continues to evolve rapidly, and both efficiency gains and access to advanced hardware will play crucial roles in shaping the future of AI development globally.
