How I route work between three AI models
# How I route work between three AI models
A few months back I sat down with my AI bill and worked out what I was actually paying for. Most of the volume was tiny jobs. Reformat a date. Decide whether a two line message reads positive or negative. Pull one name out of a sentence. Tag a note by topic. Every one of those calls was going to the biggest model I had access to, the kind built for long reasoning and careful writing, and every one was billed at that model's rates. Nothing about the work needed it. I was paying specialist prices for filing.
So I built a router. It is a small dispatch layer that sits in front of all my AI work, looks at each incoming job, and sends it to the cheapest model that can do it properly. Output quality stayed where it was. The bill dropped by more than half. This post covers how the router decides, where the savings really come from, and the ways the whole idea can turn around and bite if you stop watching it.
What routing means here
The word sounds more technical than the idea deserves. Picture the front desk at a busy help desk. Questions arrive, and the person at the counter answers almost none of them personally. They size each one up and pass it along. Routine questions stay at the counter. Odd ones go up to a specialist whose time costs more. My router is that person, implemented in about a page of code.
Behind it sit three models instead of one. A cheap fast one, a mid tier workhorse, and a large expensive one. No single model sees everything anymore. Each job gets matched to a tier, and that matching is the entire system. Calling a model is the easy part, anyone can wire that up in an afternoon. The part that pays is being honest about how hard each of your tasks really is, because most of them are far easier than they feel while you are writing the prompt for them.
The three tiers
The small model handles narrow, tightly bounded work. Sorting, labelling, simple formatting, extracting a single field from text. Anywhere the answer has little room to wander, it does the job well, at a fraction of what the other two cost per call.
The middle model is the workhorse and carries the bulk of the real work. Anything that needs some judgment or actual writing lands here, like summarising a long messy thread or drafting a first version of something I will edit myself. It costs noticeably more than the small one and still sits nowhere near the top rate.
The large model is reserved for genuinely hard reasoning, plus the jobs where a mistake is expensive enough that I want the best available shot at a correct answer on the first pass.
When I finally measured the mix, the split surprised me. Roughly seven in ten of my tasks land on the small model, and only a thin slice ever reaches the top tier. For a long time I had been paying top rates for that entire pile of easy work without noticing, because each individual call is cheap enough to disappear into the total at the bottom of the bill.
How it decides
I keep the routing rules deliberately dumb. Most of it runs off the task type plus a couple of readable signals. Jobs I have already labelled as simple go straight to the small model with no evaluation at all. A long input, or a request for reasoning or careful writing, bumps the job up a tier. A short list of tasks is pinned to the big model permanently, because experience has already shown me they are hard and I see no reason to let the router relitigate that.
I did try the clever version once, where a model examines each task and picks which model should run it. It added cost and it added delay, all to make a decision that a few plain rules were getting right almost every time anyway, so I deleted it. A router you can read and predict in ten seconds beats an opaque one, especially on the day something misbehaves and you need to know why a job went where it went.
There is a wider lesson in that, one I keep having to relearn across everything I build. The part of a system you most need to trust should be the most boring part of it. Save the cleverness for the work itself rather than the plumbing that moves the work around. Make the router smart and you now own a component that can be confidently wrong about where jobs go, which means every incident opens with two mysteries instead of one.
The safety net underneath
Sending a job to the cheap model is a bet, and sometimes the bet loses. The small model returns something garbled, or misreads the request, or produces nothing usable. So every routed job passes a basic shape check on the way out, a quick test of whether the answer is even the kind of thing this task should produce. Fail that check and the job escalates to the next tier and runs again from scratch, the same way a stuck question gets walked up the chain when the front desk cannot resolve it.
That fallback is what makes the system safe to lean on daily. If a bottom tier failure ships quietly, the bad answer goes out the door and the savings have been paid for out of quality, which defeats the entire point. The shape check turns the worst case of a wrong routing decision into one extra call and a second of added delay. I only need the small model to be right most of the time, because the net catches the rest.
What it actually saved
Routing changed nothing about per call pricing. Every model still costs exactly what it costs, all of them are paid services, and none of this is free. What changed is the mix. With one model doing everything, every task pays the top rate. With the router in place, most tasks pay the small rate, a slice pays the middle rate, and only the hard residue pays the top. When the bulk of your volume is easy repetitive work, moving that bulk down to a cheap model is where nearly all of the saving lives.
For my workload the bill dropped by more than half, and output quality did not move in any way I could detect, because the easy tasks never needed the big model to begin with. They were defaulting to it out of laziness. I want to be careful about how that sounds. The saving comes entirely from refusing to overpay for the easy majority of the work. Anyone hoping for more magic than that will be disappointed.
Where it bites back
The first cost is complexity. With three models in play, the opening move of every debugging session is working out which tier even produced the suspect output. That is a real, ongoing tax on attention, and it never goes away.
The second is drift. The small and large models do not phrase things the same way. If a task quietly starts routing to a different tier than it used to, the tone and shape of its output shifts underneath whatever consumes it, which is disorienting when you were not expecting the ground to move.
The third is the sneakiest. The fallback can hide problems. If one tier keeps failing and silently escalating, you pay for two calls on every one of those jobs and nothing ever looks broken. The mechanism added to protect quality quietly eats the savings it was protecting. So I log which tier handled every job, and every so often I sit down and compare the real mix against what I expect it to be. Routing pays for itself only while somebody is watching it. Left alone, a clever cost saver drifts into a silent cost leak, and the first you hear of it is the bill.
Should you build one
If your AI usage is small, or nearly all of it genuinely needs a strong model to do well, skip this. A single model is simpler to reason about, and the overhead buys you nothing yet. The signal to act is a bill climbing on volume rather than difficulty, paired with the realisation that a large share of your tasks are simple and repetitive.
Skip the full three tier build on day one as well, which is a mistake I nearly made myself. Start with two tiers, one cheap and one strong, and a single rule. Obviously simple work goes to the cheap model, everything else goes to the strong one. Measure what that saves over a week. That one split captured the large majority of my benefit long before the third tier or the fallback existed. The router never made anything smarter. It stopped me overpaying for easy work, and at any real volume the easy work turns out to be most of the bill.
I write up the systems I actually run, including the parts that go wrong, on the [home page](/).
Get new guides and videos first — join the Telegram channel.