AI Compute Is Scaling 5x Faster Than Moore's Law—and It's Costing Millions

Forget Moore’s Law. The real exponential to watch in artificial intelligence isn’t the density of transistors, but the raw computational horsepower being thrown at training the biggest models. A new analysis from OpenAI lays out a staggering trend: since 2012, the amount of compute used in the largest AI training runs has doubled every 3.4 months. That’s a pace that makes the old 2-year doubling period of Moore’s Law look glacial. To put it in perspective, this metric has increased by more than 300,000 times over a six-year period. A standard 2-year doubling would have yielded only a 7x increase. We’re not just in a new era; the graph is practically vertical.

OpenAI’s researchers deliberately chose to track compute per training run, not just the speed of GPUs or the size of data centers. That’s the figure that best correlates with the capability of a final model. The limitation isn’t just raw chip speed, but the brutal challenge of parallelism—how to get thousands of chips to work in concert without their efficiency collapsing. The analysis breaks this history into distinct eras. Before 2012, GPUs were an oddity in machine learning. From 2014 to 2016, we saw the first serious scaling to 10-100 GPUs, before diminishing returns on sheer data parallelism kicked in. The real explosion happened after 2016, driven by novel approaches to algorithmic parallelism like massive batch sizes and architecture search, paired with specialized hardware such as TPUs and faster interconnects.

What’s propelling this isn’t magic, but money and engineering. Custom silicon, like Google’s TPUs, offers more operations per second for every dollar. But the dominant force has been researchers figuring out how to lash more chips together and organizations writing the checks to make it happen. AlphaGo Zero is the poster child, but the analysis hints that many other applications at this scale are likely already running behind closed doors in production environments. The report estimates the hardware for today’s largest training runs costs in the single-digit millions of dollars to purchase outright, though the amortized cost is significantly lower.

Will this rocket ship stall out? Cost and physics are the obvious barriers. But the authors argue it would be a mistake to bet on a slowdown in the short term. The crucial point is that most neural net computation today is still spent on inference, not training. This means companies can reallocate or purchase massive fleets of chips for training if the economic incentive is there. With a wave of AI-specific hardware startups promising dramatic efficiency gains and a global hardware budget of $1 trillion a year, the absolute ceiling is still distant. The real question isn’t whether we’ll hit a wall, but what it means to have systems whose capabilities have been amplified by a factor of 300,000 and are now doubling several times a year.