The market is obsessed with GPT-5 and Claude 4. The money is in models you can run on a phone.
Small language models — SLMs, broadly defined as models under 10 billion parameters — are solving 80% of enterprise AI use cases at roughly 5% of the inference cost of frontier models Source. They run on-premises, on-device, behind the corporate firewall, and on edge hardware that costs less than a month of frontier API credits. The frontier labs are burning billions on training runs that produce models 99% of enterprises will never need. The real AI deployment is happening at the edge, and the numbers are starting to tell the story.
Microsoft's Phi-3-mini has 3.8 billion parameters and was designed specifically for on-device deployment Source. It scores competitively against models 20 times its size on enterprise benchmarks — summarization, classification, extraction, structured output. The Phi-3 family includes even smaller 1.5B-parameter variants. A 3.8B-parameter model runs on a laptop GPU, a Jetson edge device, or a server-class CPU without breaking a sweat.
Meta's Llama 3.2 shipped 1B and 3B models optimized for edge and mobile devices in September 2024, both supporting 128K token context windows Source. Apple published its on-device foundation model architecture the same year, using low-bit palletization to achieve the memory footprint required for iPhone inference Source. That's three of the largest technology companies on Earth betting that the market for inference is not "one massive model in the cloud" but "thousands of small models on devices and behind firewalls."
The enterprise economics are what make this structural rather than cyclical. A frontier model API call costs something like $15 per million tokens. A 7B-parameter model running on a single A100 behind the firewall costs maybe 30 cents per million tokens in amortized hardware. An 8B on two CPU cores costs pennies. For document classification, entity extraction, customer support routing, and the ninety other enterprise AI tasks that are more "deterministic reasoning" than "creative generation," the small model is indistinguishable from the frontier model in output quality and an order of magnitude cheaper.
Here's the colloquial version nobody's putting in investor decks: every enterprise that deploys a fine-tuned 7B-parameter model behind its own firewall is inference revenue that doesn't go to a hyperscaler. Multiply that by the Fortune 500. The hyperscalers know this, which is why Azure markets Phi-3 as an Azure service even though the model is small enough to run locally — the cloud vendors want a piece of every inference call no matter where it runs. But the fundamental physics of inference cost favors local deployment for anything that doesn't require the frontier model's reasoning capacity.
The fine-tuning ecosystem is the accelerant here. The Hugging Face model hub now hosts thousands of fine-tuned variants of Mistral, Llama, Phi, and Qwen models. Mistral, the French AI lab, ships its fine-tuning tools as open source alongside its models, enabling developers to adapt 7B-class models to domain-specific tasks with consumer-grade hardware Source. A mid-size enterprise can take an open-source 7B model, fine-tune it on 10,000 internal documents, and deploy it in a weekend. The total inference cost for a year of enterprise use is less than a single month of GPT-5 API calls at moderate volume. Self-hosted LLM deployments running on dedicated infrastructure offer a fundamentally different cost structure than per-token API billing — the hardware is a sunk cost, not a variable one Source. The accuracy on domain-specific tasks — contract review, compliance classification, technical support — beats the frontier model because the fine-tuned model has seen the enterprise's actual data.
The AI inference market itself is projected to reach meaningful scale by 2027, driven largely by on-device and edge deployment rather than centralized cloud inference, according to MarketsandMarkets research Source. This aligns with the deployment pattern we're seeing: frontier models for the hard one-off tasks, SLMs for everything that runs continuously. The economics don't just favor this pattern. They mandate it.
The AI industry is running a two-track experiment. Track one: can we build models that approach human-level reasoning across arbitrary domains? Track two: can we make existing model capabilities cheap enough to run on a toaster? Track one gets the magazine covers. Track two gets the purchase orders. The value of AI is the sum of what enterprises actually deploy, not what the labs can demonstrate in a blog post. Zero enterprises are running GPT-5 on a phone. A hundred million phones will run on-device models this year.
The next time someone tells you about a frontier model's MMLU score, ask them what it costs to run inference. Then ask them whether a 7B fine-tune could do the same job for 5% of the price. The answer to the second question is the only one that matters for the enterprise P&L.