The artificial-intelligence buildout has acquired a wonderfully specific price tag: $401 billion in new infrastructure spending during 2026, meaning CIOs are committing factory-scale capital to equipment many of their teams still cannot keep busy Luby. The provenance is less satisfying. The figure comes through an industry consultancy’s interpretation of Gartner forecasts, not a clean Gartner spending category, which is a useful reminder that AI accounting has become nearly as creative as AI marketing.
Still, the underlying problem is real.
Bessemer Venture Partners counts 190 gigawatts of announced hyperscale capacity across 777 projects as of early 2026, meaning utilities and developers are planning an electrical system measured less like corporate IT and more like national industry Bessemer Venture Partners. Roughly 148 gigawatts remain in planning, while about 21 gigawatts are under construction, meaning most of the promised capacity still exists in spreadsheets, permitting offices and power queues rather than server halls Bessemer Venture Partners. JLL estimates global data-center capacity could reach 200 gigawatts by 2030, enough to turn power procurement into a board-level constraint rather than a facilities chore JLL. Big numbers. Long cables.
The contradiction is that GPUs can be both scarce and poorly used. One enterprise survey compilation claims 86% of organizations running GPUs report utilization below half, meaning expensive accelerators spend more time waiting than many owners would admit in an investment committee memo Luby. Treat the precision cautiously; vendor-sponsored utilization statistics tend to arrive with a product conveniently attached.
Why can scarce GPUs sit idle? Because scarcity exists at the fleet level, while waste occurs inside individual clusters: jobs request entire accelerators when they need fractions, memory allocations strand compute, training runs block inference queues, notebooks reserve capacity overnight, and data pipelines leave processors waiting for input. Kubernetes can discover accelerators through device plugins, but discovery is not the same thing as efficient scheduling across heterogeneous chips, clouds and organizational budgets Kubernetes Documentation. Slurm handles generic resources and remains deeply embedded in technical-computing environments, yet enterprise AI estates increasingly combine batch training, interactive development and latency-sensitive inference in ways that make static queues feel rather old SchedMD.
Not great.
That gap has created a market for GPU-orchestration software: schedulers, workload managers and control planes that decide which job runs where, when capacity should be split, and whether work should spill from an internal cluster to a cloud provider. The engineering is unglamorous (which is usually a good sign), involving queue policy, checkpointing, topology awareness, memory constraints, identity controls and cost attribution across machines that may not share drivers, networking or failure behavior. You can see where this is going.
The best products do more than draw a prettier utilization dashboard. They pack workloads around memory and latency requirements, suspend low-priority jobs, shift training runs toward cheaper capacity and give finance teams a defensible chargeback model—turning “AI infrastructure” from a pooled science project into something resembling an operating system for capital.
Nvidia’s Multi-Instance GPU technology can partition a supported accelerator into as many as seven isolated instances, meaning several developers or inference services can use hardware that might otherwise be monopolized by one lightly loaded job Nvidia. Orchestration vendors build above such features, matching jobs to slices and reallocating them as demand changes. Credit where it’s due — this part is difficult to build well.
But investors should resist treating orchestration as an uncontested software control point. Nvidia, cloud providers and Kubernetes distributors all have incentives to absorb scheduling into their platforms, especially when better utilization can delay a customer’s next hardware order or, depending on pricing, encourage more consumption. Nvidia’s move to acquire Run:ai made the strategic direction plain: the company selling the shovels also wants to manage who gets one Nvidia.
The investable question is therefore not whether enterprises need scheduling. They do. It is whether an independent vendor can own enough policy, telemetry and workload portability to avoid becoming a feature inside the accelerator stack.
This is a mistake: valuing GPU orchestration like ordinary infrastructure software before establishing who controls placement decisions. Gross margins may look familiar, but the bargaining structure is not; the supplier of scarce silicon, the operator of scarce power and the owner of the workload are each trying to capture the same efficiency gain. Three parties. One idle chip.
The winners will sit where technical scheduling meets financial authority, telling engineers when a job can run and telling CFOs why another cluster need not be ordered yet.
Every unused GPU is a tiny, very hot capital-allocation memo.