Most production AI tasks are routine, and a new class of small models handles them at a fraction of the cost. The frontier models are becoming the exception, not the default.
A quick read: the essentials, fast.
For two years the story of AI was a race to build the biggest model. The signal now points the other way. Teams shipping real products are quietly routing most of their traffic to small models, the kind that run cheaply and answer in a fraction of a second, and reserving the expensive frontier models for the handful of requests that actually need them.
This is not a downgrade. It is a recognition of what most AI workloads actually look like once they leave the demo and meet real users.
Most requests are boring#
If you instrument a deployed AI feature and look at what users actually ask, a pattern shows up fast. The bulk of requests are short, repetitive, and shallow: classify this ticket, extract these fields, rewrite this sentence, decide whether this comment is spam, tag this photo, summarize this paragraph. None of that needs a model that can also pass a law exam or write a working compiler.
A rough split most teams recognize:
- A large majority of calls are routine and well within a small model's reach.
- A smaller slice needs real reasoning, long context, or careful synthesis.
- A thin sliver is genuinely hard and worth paying frontier prices for.
When the cheap tier can absorb most of the volume, the economics change completely. You are no longer paying premium rates for a thousand requests to find the ten that needed it. You pay premium only for the ten.
Why this is happening now#
Small models got good enough. A few years ago a compact model was a toy that hallucinated through anything past trivia. The current generation of small open-weight and efficient models handles classification, extraction, summarization, and simple drafting at a quality that clears the bar for production use, and it does so fast enough that the speed itself becomes a feature.
Three forces pushed this:
- Better training data and distillation, where a large model effectively teaches a small one to imitate its answers on a narrow band of tasks.
- Hardware and serving stacks tuned for cheap, fast inference at high volume.
- Teams getting honest about how little reasoning most of their tasks actually require.
The result is that the floor moved. The worst a capable small model does today is roughly where a much larger model sat not long ago, and for routine work that is more than enough.
The router is the real product#
The interesting engineering is no longer the model. It is the routing layer that decides which model gets each request. Send the easy ones to the cheap tier, escalate the hard ones, and you get most of the quality of a frontier model at a small fraction of the cost.
The competitive edge is shifting from which model you can access to how intelligently you decide when to use it.
That routing logic, done well, is worth more than access to any single model. It is also where most of the cost savings live, and it is the part competitors cannot copy by simply buying the same API key. A good router learns which requests tend to need escalation, fails over gracefully when the cheap tier is unsure, and keeps a clear record of why each call went where it did.
What this signals#
The "bigger is always better" framing was always a half-truth aimed at benchmarks, not at production. The market is now sorting models into tiers the way every mature compute market eventually does: a cheap default for the common case, premium capacity for the hard case. Expect pricing, tooling, and product design to organize around that split rather than around a single flagship model. The teams that win will not be the ones with access to the biggest model. They will be the ones who almost never need it.
Frequently asked questions
Why are teams routing most AI traffic to small models?
Most production requests are short, repetitive, and shallow (classifying tickets, extracting fields, rewriting sentences, tagging photos, summarizing), which small models handle well. They run cheaply and answer in a fraction of a second, so frontier models are reserved only for the handful of requests that truly need real reasoning.
What kinds of tasks can small models handle in production?
The current generation of small open-weight and efficient models handles classification, extraction, summarization, and simple drafting at a quality that clears the bar for production use, and fast enough that the speed itself becomes a feature.
Why have small models become good enough only now?
Three forces pushed it: better training data and distillation (a large model teaching a small one to imitate its answers on narrow tasks), hardware and serving stacks tuned for cheap fast high-volume inference, and teams getting honest about how little reasoning most tasks actually require.
What is the routing layer and why does it matter?
The router decides which model gets each request, sending easy ones to the cheap tier and escalating hard ones. Done well it delivers most of a frontier model's quality at a small fraction of the cost, and competitors can't copy it by simply buying the same API key.
How does using small models change AI economics?
When the cheap tier absorbs most of the volume, you no longer pay premium rates for a thousand requests to find the ten that needed it. You pay premium only for those ten, which changes the economics completely.
Discussion