Is the AI Infrastructure Boom Built on a False Premise?

When Arm Holdings chief executive Rene Haas told CNBC last week that much of AI's future workload should migrate from cloud data centres to personal devices, it was easy to dismiss it as self-serving commentary. After all, ARM's business model depends on selling energy-efficient chip designs for phones, laptops, and wearables, precisely the devices Haas envisions running AI inference at scale.

But Haas may have inadvertently articulated what could become the first serious challenge to the industry's most expensive bet: That artificial intelligence demands an endless expansion of hyperscale cloud infrastructure. If he's right, and computing history suggests he might be, the implications for the $1 trillion web of OpenAI deals now reshaping Silicon Valley could be severe.

The ARM CEO's argument is deceptively simple. While AI model training will remain the province of massive data centres, the far more common task of running these models, called inference, can increasingly be handled locally. History, he notes, has consistently moved toward hybrid computing models, balancing centralised and distributed processing. What makes this observation more than just corporate positioning is the sheer scale of infrastructure investment now predicated on the opposite assumption.

Consider the financial architecture currently taking shape. OpenAI has committed to hundreds of billions in data centre deals, anchored by a $300 billion contract with Oracle for 4.5 gigawatts of computing capacity. Nvidia is investing up to $100 billion in OpenAI, capital that will largely circle back to Nvidia as OpenAI purchases its chips. AMD recently secured its own multibillion-dollar OpenAI deal, positioning itself to finally challenge Nvidia's dominance. This entire build-out rests on a single premise, that AI workloads will overwhelmingly require centralised cloud processing.

But inference, not training, is where the real demand growth lies. Training large language models is eventually limited and resource-intensive. Inference, serving AI to billions of daily queries, is what scales exponentially with adoption. If edge devices can handle much of this workload, as Haas suggests, the projected energy and infrastructure requirements could prove overstated.

The potential consequences of this become clear when examining OpenAI's economics. The company has unveiled a five-year, $1 trillion infrastructure spending plan while generating just $13 billion in annual revenue, 70 % from consumers paying $20 monthly for ChatGPT. It burned through $8 billion in operating losses in the first half of this year alone, excluding capital expenditure on chips and data centres.

When asked if 10 times more computing power would yield 10 times more revenue, OpenAI president Greg Brockman’s answer was rather vague. "I don't know if we'd have 10 times more revenue, but I don't think we would be that far", he said. I wonder if OpenAI would really be able to seemingly 8x their revenue just because they have more computing. Just recently, Deutsche Bank published research showing the growth of ChatGPT’s paying users stalling in Europe. If clear, value-creating business applications of AI continue to lag, such predictions rest on shaky ground. Or as Deutsche Bank warned:

“If the slowdown in paying-user growth continues, the valuation framework for the entire AI industry may face a major recalibration.”

Importantly, the financial vulnerabilities within this AI ecosystem are not evenly distributed. Nvidia and AMD are funding their AI bets through equity and cash flow, expensive in terms of dilution and opportunity cost, but not existentially risky. Oracle, however, has taken on $18 billion in debt to finance its AI data centre buildout, with Moody's already flagging concerns about the concentration risk from its OpenAI relationship. If the anticipated cloud workloads fail to materialise, or progress more slowly than forecast, Oracle could find itself dangerously overleveraged, holding vast, underutilised infrastructure.

The market has not yet fully absorbed this risk. Oracle's stock has surged on the strength of its AI contracts, with analysts focused on the revenue upside while downplaying execution and concentration risks. Moody's warning about Oracle's $300 billion backlog seems to have been largely dismissed as overly conservative. But if edge inference takes a meaningful share from cloud processing, Oracle could become the company left holding the infrastructure bag when the music stops.

Haas's edge inference thesis introduces a new variable into this calculus. If local processing can handle even 30 to 40 per cent of the workload currently allocated to cloud infrastructure, the "total addressable market" that justifies today's valuations contracts sharply. AMD's projected chip sales to OpenAI might not materialise at forecast levels. Nvidia's circular $100 billion investment might generate less returning revenue than modelled. Oracle's debt-financed data centres could sit partially idle, generating insufficient cash flow to service obligations.

While the technology is real, the applications are valuable, and the long-term trajectory remains transformational, the current investment cycle may be built on assumptions about infrastructure demand that are more fragile than commonly understood. Haas was talking his own book, certainly. But he was also presenting a different side of the AI buildout story. One that might lead to billions of CapEx dollars not seeing much ROI. The question is whether anyone committing these billions is listening.