The Routing Paradigm for Enterprise AI
Routers — the systems that decide, query by query, which AI model answers — look like a cost-saving feature. They are not. They are the AI stack growing a price-discovery mechanism, and the signal they generate cascades through all nine layers of the Map of AI: down to compute, silicon, and energy, and up to distribution and governance.
The inversion: routers turn models from products into suppliers, migrating pricing power from the model layer to the allocation layer — whoever makes the purchasing decision captures the leverage.
The pincer: labs compress inference costs from the supply side (compute multipliers) while routers compress demand from the buy side; the gains land in different pockets.
The cascade: routing reshapes every layer beneath it — extending GPU fleet life, redirecting silicon roadmaps toward inference-specialized chips, and turning energy allocation into a per-query decision.
The barbell: optimizing allocators hollow out the mid-tier model market, splitting demand between irreplaceable frontier intelligence and near-free commodity models.
The meta-moat: evaluation data — knowing empirically which model is good at what — is the routing layer’s defensible asset; the scoreboard beats the switch.
The routing paradigm is not a product story. It is the moment the AI stack acquired an internal market — a mechanism that prices intelligence continuously and transmits that price signal down to the physical world and up to the regulatory one. Whoever operates that mechanism operates the stack’s capital allocation.
The Architecture: A Market Grows Inside the Stack
Follow me also on The AI Supercycle!
Strip away the product names and a router is one thing: a capital allocation engine for intelligence.
Every query entering an AI system carries two hidden variables — the difficulty of the task and the value of getting it right. For three years the industry priced neither. Users manually selected a model, defaulted to the most expensive one out of habit or brand trust, and paid frontier prices for tasks a model one-tenth the cost could handle. Structurally, this was a market without price discovery: heterogeneous demand met heterogeneous supply through a single crude interface — the model picker dropdown — and the mismatch was absorbed as waste.
Routers are the price discovery mechanism arriving. They estimate task difficulty, match it against a live map of model capabilities and prices, and dispatch each query to the cheapest supplier that clears the quality bar. The Information’s reporting this week catalogued the forms this now takes, and the taxonomy is worth holding onto because each form represents a different owner of the allocation decision:
Lab-internal routing — OpenAI’s GPT-5 switching between sub-models under the hood. The lab keeps the allocation decision in-house and arbitrages its own portfolio. The customer never sees the spread between what they paid for and what actually ran.
Neutral marketplace routers — OpenRouter, which raised $120 million on the thesis, plus Martian and Not Diamond underneath it. These sit outside any lab and optimize honestly across the full supplier population.
Platform gateways — Databricks’ Unity AI Gateway, Palantir’s Evolve. Enterprise infrastructure players bundling routing into the data and workflow platforms enterprises already run on.
Agent-level routers — Cognition’s new sidekick architecture, which delegates easier subtasks to cheaper agents mid-workflow. Here routing happens inside a running agentic process, not at the query boundary.
DIY and model-as-router — IT departments vibe-coding their own switches, or developers handing a cheap model like DeepSeek a menu of models and asking it to pick. The Arcee AI approach: use the cheapest competent judgment available to allocate the expensive judgment.
Five forms, one mechanism: a decision that used to belong to the user — or default silently to the model vendor — now belongs to an intermediary layer. And in business architecture, whoever makes the purchasing decision captures the negotiating leverage.
The reported numbers establish that this is not marginal optimization. Palantir says Evolve cut computing costs for one task by 97% by swapping a frontier OpenAI model for its Nano sibling. McCarthy Building, a $9 billion construction firm, cut token consumption 60% year-over-year — partly through model swapping, partly because the router rewrites prompts to burn fewer tokens for the same outcome, which is a second and underappreciated function: routers optimize not just which model but how the model is asked.
Cognition’s router reportedly matched Anthropic’s Fable 5 on a coding benchmark at 35% lower cost. Sakana claims a coordinated multi-model system competitive with frontier flagships by routing math to one lab and science to another.
When an intermediary reliably delivers frontier-equivalent outcomes at a third to a thirtieth of the cost, it is not a feature. It is a new layer of the stack asserting itself. Which raises the structural question this piece exists to answer: where exactly does it sit, and what does it do to everything above and below it?
