You don't always need the most expensive model

This article was originally written in Dutch.

‍

There’s a reflex in most teams: when it gets serious, you grab the most powerful, most expensive model. The frontier feels like the safe choice. Who deliberately picks less?

First, look at what “less” means these days. Claude Haiku 4.5, Anthropic’s small, fast model, matches the larger Sonnet 4 on coding, computer use and agent tasks according to Anthropic, and scores 73.3% on SWE-bench Verified. One customer measures that it reaches ninety percent of the much larger Sonnet 4.5. Those are partly Anthropic’s own numbers, so read them with a grain of salt. The prices, though, are hard: on Anthropic’s own API the frontier model Fable 5 costs ten dollars per million input tokens and fifty per million output. Haiku does the same work for one and five. A factor of ten, within a single vendor, for a model that half a year ago would have been state of the art.

In other words: the model that’s called “cheap” today was yesterday’s frontier. And you’re paying the frontier price out of habit.

The wrong question

The discussion is almost always about which model is best. That’s the wrong question. The leaderboards measure breadth: can the model hold forth on Shakespeare, quantum physics and every Lego set. Your product doesn’t do that. It does one thing, or a handful of things, and it does them all day.

It’s becoming a bit of a hobbyhorse of mine (see the piece on tokens getting too expensive): the winning product doesn’t run on the biggest model, but on the smallest one that does its one task exceptionally well. Since I wrote that, a few things have shifted: there are more real cases, and the small general models have become good enough for most production work. The builders do it themselves too: Anthropic recommends routing easy, common questions to a cheap model and sending only the hard cases to a more expensive one.

Cheap is no free lunch

And yet “always cheap” is as lazy as “always expensive”, because cheap shifts the cost. Not to your token bill, but to your attention.

At our hackathon one team deliberately chose Haiku and got just as far with it. Only: they resorted to mob programming to keep up, because the smaller model takes the wrong assumption sooner and cheerfully builds on it. I used Gemini myself (I was out of Claude credits) with copy/paste into Python. That worked, but slower, and it worked precisely because copy/paste kept everything small and predictable. Bigger models have their own trap: Gemini tends to trust its own overview a little too much.

I saw it sharpest on another project, built with an agentic method. At some point it got so complex that it couldn’t be debugged anymore. The way out wasn’t a bigger model, but a small test program we used as input for the bug fix. Exactly the engineering practices from an earlier post: small, testable, verifiable. What my colleague Michiel Kooiman calls automating your assumptions simply happens a touch faster with a cheaper model. It’s manageable, but it’s no free lunch.

The right model is often small, sometimes regional

Match, then, don’t minimize. The most expensive model earns its place when exploring and on tasks that need broad, genuine reasoning. But for a bounded task, specialized beats generalist, and not by a small margin.

Mistral shows this with Saba, a 24-billion-parameter model trained on the Middle East and South Asia. According to Mistral it gives more accurate answers than models more than five times its size, and does so faster and cheaper. It’s particularly strong in South Indian languages like Tamil, runs on a single GPU, and can be deployed inside your own secured environment. It serves markets the United States shows little interest in and where China is viewed with some suspicion. What Mistral does deliberately there, you can do too: pick the model that fits your application. That may be a Mistral, a Haiku, or an open model you run yourself. The art is in the match, not the brand.

The bill at scale, and who bends under it

At a few tens of euros a month this is an academic discussion. But a serious product easily runs to a thousand euros a day, and then that factor of ten to a hundred between frontier and small model stops being a detail and becomes your biggest variable cost.

How strong that pressure is, you can see at Airbnb. CEO Brian Chesky publicly called the Chinese models fast and cheap, and his company runs on Alibaba’s Chinese Qwen. That a large American company says so out loud, right up to the United States Congress, shows this cost advantage is no fringe phenomenon. And a second motive comes in here: done smartly, running your own infrastructure buys you not just cost but data control. For a Dutch bank, hospital or government body, that’s no side issue.

The picture I’d work toward is an orchestration in two layers: an expensive, smart model (a Fable or a Sonnet) that makes the hard decisions and divides the work, and beneath it a legion of cheap open models (think Mistral, Qwen or Gemma) doing the bulk, on your own hardware or in your own image. The smart direction you rent, the mass you run yourself.

I’d recommend that today already, with one important nuance: it’s about optionality. Make sure you can switch to your own infrastructure when cost or data requirements demand it. You don’t have to do it right away, but your architecture has to allow it. Whoever locks themselves fully into one expensive API discovers at the worst possible moment that switching isn’t a button.

And the choice isn’t binary. Between an expensive API and your own server rack sits a middle way: running open models with a hosted provider. You get part of the cost and control gains without managing a GPU park yourself.

Because be honest about that price. Running a model yourself isn’t a button but a competence: hardware, monitoring, updates, and someone to take over when that one colleague is on holiday. As I wrote in the tokens piece, ownership without capability isn’t control but a new risk. Count those people in, not just the tokens. And the same goes for babysitting a cheaper model: what you save on the bill, you can lose again in attention.

How you dare to choose cheaper

That you dare to scale down to a cheaper model hangs on one thing, and it ties back to those engineering practices: you have to be able to demonstrate it’s good enough. Without a check, a test set, an evaluation, model choice is a gamble, and then you grab the most expensive one out of uncertainty. With a check it becomes a measurement. In practice that’s not a big project: a small, task-specific test set of some twenty real cases with the answer you expect, which you run two models against side by side. The result tells you whether the cheap model suffices. The test suite that lets you trust AI code is the same one that lets you scale down without fear.

The bottom line

The most expensive model should be a fallback, not a starting point. Reverse the habit: start with the cheapest model that could plausibly handle the task, test it against your own check, and scale up only when it fails that test. And calculate the success scenario, not the pilot.

That takes three things that have nothing to do with the brand of your model: knowing what your task actually needs, a way to measure whether it works, and an architecture that gives you the freedom to switch. Whoever has that in order usually finds that “good enough” is a good deal cheaper than assumed. And whoever doesn’t pays the frontier price for peace of mind, not for results.

‍