Executive summary

What is documented without dispute: prices are rising even as idle capacity is reported. AWS raised H200 instance pricing from $34.61 to $39.80 an hour on January 4, 2026, without announcement, as described in VEXXHOST's account of the GPU capacity crunch. Meanwhile hyperscalers are building their own chips: custom ASIC shipments are projected to grow 44.6% in 2026 against 16.1% for GPU servers, per Startup Fortune's reporting on custom AI chip growth, and Alphabet briefly surpassed Nvidia's market cap in May 2026, a rally CNBC attributed to owning "most of the stack," in CNBC's coverage of Alphabet's 160% rally.

  • Chip demand and reported idle capacity are not mutually exclusive; the evidence suggests enterprises are reserving GPUs defensively at rising prices rather than buying only what they can run.
  • Capital markets are already rewarding vertical integration (chips plus software plus distribution) over raw GPU accumulation.

The open question for a board reviewing its own AI infrastructure spend: has anyone measured actual GPU utilization against the reserved capacity on the invoice, or is the number simply unknown.

For practitioners

WEKA's own account of the mechanism is more specific than the headline gap it produces. The vendor attributes idle GPU time to two separate failure modes that happen to compound. The first is a data pipeline problem: GPUs sit idle waiting for training data to arrive from storage, because throughput to the accelerator is gated by I/O rather than by compute. The second is a memory problem specific to inference: when a GPU runs out of high-bandwidth memory mid-generation, it has to recompute tokens it has already produced instead of retrieving them, which burns cycles a well-provisioned system would not spend at all. WEKA's analysis of AI storage architecture treats both as symptoms of the same design mistake. Storage and memory are managed as separate tiers rather than one system co-located with the compute it feeds.

That framing matters for how a platform team reads its own cluster metrics. The fix WEKA proposes, software-defined storage co-located with GPU servers, extending HBM to flash, implies the bottleneck is measurable at the storage layer specifically, not just as an undifferentiated "GPU is idle" reading. Whether that diagnosis generalizes past WEKA's own customer base is not something this record can confirm.

The Kubernetes-level figure from Cast AI implies a different failure mode entirely: allocation rather than pipeline stalls. A cluster averaging 5% utilization across 23,000 clusters, as Rack2Cloud's analysis of enterprise GPU waste describes it, is consistent with GPUs requested via resource limits and then held by pods that aren't running dense workloads continuously.

That is the scheduling-and-reservation problem, not the feed-the-accelerator problem WEKA describes.

WEKA's fix is a storage re-architecture: colocating NVMe-backed software-defined storage with the GPU fleet, which is a capital and integration project. The reservation problem is closer to a scheduling and chargeback fix: MIG partitioning, bin-packing jobs onto shared GPUs, or exposing per-job utilization back to the requester so the cost of holding idle capacity is visible before the invoice arrives. Neither source in this record measures which failure mode dominates, or in what proportion, across the fleets each is describing.

Deep dive

Dragrush AI's press release on CoreWeave's liability structure describes roughly $35 billion in total liabilities against 1.5 gigawatts of active data center capacity by Q2 2026, financed through data center leases running 15 years, covered by debt structured on a 5-year term, backstopped by customer contracts that run 3 years. That maturity mismatch — 15 against 5 against 3 — means CoreWeave has to keep re-selling capacity into new contracts faster than its debt amortizes, which makes fleet-wide utilization a cash-flow question rather than an efficiency one. If even the lower end of WEKA's idle-capacity range applied to CoreWeave's fleet, the company would be carrying debt service on capacity that is not generating revenue during a period when its own contracts are shorter than its financing. The press release does not disclose CoreWeave's actual utilization figure, and it comes from a company selling an orchestration product into exactly this problem, so the $35 billion figure should be read as a description of exposure to a utilization shortfall, not evidence that the shortfall exists at any specific level inside CoreWeave's fleet.

The custom silicon numbers cut against the vendor-optimization narrative in a way worth stating plainly. If hyperscalers believed third-party orchestration and storage software could resolve idle GPU capacity at the levels Cast AI and WEKA describe, the economically simpler move would be to buy that software and keep renting Nvidia GPUs, rather than spend years and design cycles building Trainium, Maia, MTIA and Ironwood. Quartz's reporting on the six companies building custom AI chips shows Google and Amazon already shipping at scale, with Microsoft and Meta moving chips into production and only OpenAI and Tesla still at the design stage. That six-company commitment is consistent with a different diagnosis than the one WEKA and Cast AI offer: that the utilization problem, to the extent it is real, is bound up with the general-purpose nature of Nvidia's architecture and the margin Nvidia extracts on it, not simply with the absence of a co-located storage layer or a smarter Kubernetes scheduler. A hyperscaler that owns the chip, the interconnect, and the scheduling stack can tune utilization internally in ways a Cast AI add-on cannot reach, which would leave the independent optimization vendors addressing a shrinking slice of the market — the enterprises and neoclouds still buying merchant GPUs — rather than the hyperscalers with the largest fleets.

Nvidia's own numbers offer one more constraint on how far the utilization story can be pushed. Data Center revenue of $62.3 billion in the fiscal fourth quarter, up 75% year over year, with quarterly gross margin at 75.0%, per Nvidia's Q4 and Fiscal 2026 results filing, is not a figure that separates GPUs sold into workloads with high utilization from GPUs sold into defensive reservations. Nvidia recognizes revenue on shipment, not on the buyer's subsequent occupancy rate, so a 5% or a 50% utilization figure downstream is invisible to this filing by construction. What the filing does establish is that the rate of chip acquisition has not slowed as the utilization reporting has accumulated through 2026 — full-year revenue growth of 65% describes a market still in an acquisition phase, whatever is happening inside the fleets already purchased. Whether that gap closes because enterprises start measuring what they are running, or because the hyperscalers' custom silicon absorbs enough of the workload that the merchant-GPU utilization question becomes someone else's balance sheet problem, is not something the filing or either vendor's telemetry can settle.