How I keep my AI costs under control
# How I keep my AI costs under control
The first month I ran AI seriously, the bill surprised me. Nothing catastrophic. Just enough that I sat down and worked out what I was actually paying for, line by line. Until then I had treated API calls as basically free, a rounding error I could ignore, and for light usage that attitude survives contact with reality. Then you start building for real: loops, batch jobs, long running conversations. The math changes fast.
What I found was a handful of small habits stacking quietly rather than one big leak. These are the specific changes that stopped the surprises, and the principles behind them that I now apply to everything I build.
What the meter is actually measuring
The unit is a token, roughly four characters of text. Billing is tokens in plus tokens out. Input covers everything you send: the system prompt, the conversation history, any documents attached. Output is whatever comes back, and on nearly every provider I have used, output tokens run two to five times the price of input tokens. A chatty model costs more than a terse one doing the same job.
Two multipliers sit on top of that. Price per token climbs steeply with model size; a model four times as capable can cost ten times as much per token. And context gets billed on every single call. Carry a ten thousand token context through twenty calls and you have paid for those ten thousand tokens twenty times.
Loops are the silent multiplier
The cost that bit me hardest early on was never one expensive call. It was calls happening more often than I realised. A retry loop that ran six times instead of once. An agent chain taking five more steps than the task needed. A script catching an error and quietly starting over from the top. Every individual call looked reasonable. The total did not.
Agentic code is where this gets dangerous, because the model is invoking itself or other models in a chain. One runaway loop I left running overnight cost more than a full week of my normal usage. It was doing real work on every step. It just never found a reason to stop, so it kept going until I looked at it the next morning. That night changed how I instrument everything.
Use the smallest model that does the job
The change that moved the needle most was also the least clever: I stopped sending every task to the strongest model available. Defaulting to the top model felt safe, like buying insurance on output quality. Most of my workload never needed it. Summarising a short document, pulling a field out of structured text, sorting inputs into five categories: I tested these on a smaller model and the results matched the big one at a fraction of the price.
Now I run a rough tier list in my head. Small models take classification and extraction. The mid tier takes drafting that needs some judgement. The full model is reserved for the hardest reasoning and writing. The specific models rotate constantly as new ones ship, so I hold the habit rather than the names: the first question on any new task is which tier it actually needs.
Send less context
Context management is where the largest amount of quiet waste hides. Everything you send gets billed on every call, so an unbounded pile of history is an unbounded bill. I have watched setups where a chatbot reaches mid-conversation carrying twenty thousand tokens of history that neither the user nor the model needs, with every new turn adding to the pile and every call paying for all of it.
The fix is trimming, applied without sentiment. Conversation history gets a rolling window instead of the full log. Documents get searched first, and only the relevant chunks travel; the full document stays home. The system prompt stays tight, and I reread it every few weeks to cut whatever has drifted in. None of this cost me quality anywhere I could measure. A model does not need last Tuesday's conversation to answer today's question.
Cache the parts that never change
Prompt caching is the highest yield tactic on this list if your workload repeats. Most providers now discount tokens in a prefix they have seen before, which means a large fixed system prompt should not be billed at full price hundreds of times a day. A few of my workflows send the same heavy prefix on every call; switching those to cached prompts cut the input cost on that portion to a small fraction of what it was.
The catch is that caching wants identical content. Insert per-user data into the middle of the prompt and the cache breaks from that point on. Structure the prompt with the stable material first and the dynamic material last. That single layout change is most of the game; it noticeably raised my cache hit rate the week I made it.
Batch the work that can wait
Most providers also run a batch lane: submit a pile of requests, collect results within a few hours, pay roughly half the normal rate. Useless for anything user-facing. For background work it is close to free money.
I split jobs into two lanes early in every pipeline now. Whatever needs an answer in seconds stays synchronous. Whatever can wait, analysis, classification, bulk generation, goes into the batch queue. At hundreds or thousands of calls a day the difference stops being small.
When a local model earns its keep
I keep a machine with a graphics card running a small open source model for the work where quality demands are modest: rough outlines, text cleanup, filtering a long list down before the interesting items go to a smarter hosted model. Per call, the hosted price for that work is tiny. At thousands of calls a week, the local box wins.
I want to be straight about the trade, though. Local is a different cost structure rather than a free one. You pay up front in hardware, then continuously in electricity, setup time, and the occasional afternoon spent keeping it all working. A few hundred calls a month does not justify any of that; just pay the API. Tens of thousands of calls changes the arithmetic. Run your own numbers rather than taking anyone's word on it, mine included.
Ceilings first, alarms second
The tactic I wish I had set up on day one is the hard spend cap. Every provider I use can enforce a ceiling where calls start failing once you hit it. Failing calls sound scary. A surprise bill after a runaway loop is scarier, and I have had exactly one of those and zero since.
I set monthly ceilings per project and per API key, with soft alerts around seventy percent so a problem surfaces while there is still time to look into it. The ceiling represents the most I am willing to lose in a month, deliberately above expected spend, because expected spend drifts and the ceiling must not. Knowing no bug can burn past that line is worth more calm than I expected.
Ten minutes a week on the dashboard
A cap alone is a blunt instrument, so I also look at the usage dashboard once a week, same time every week, as part of a short review of everything I have running. The absolute number matters less than the shape. Is one project trending above last week. Did the integration I shipped on Tuesday show up as a spike. Is something I believed was off still quietly running.
Most weeks the answer is no news. Two or three times the check has caught a real problem early, and those catches paid for every boring ten minute session many times over.
The two patterns I audit first
When something runs expensive, I look for the same two shapes before anything else.
The first is a huge, growing context sent on every call: chatbots appending the entire message history each turn, document pipelines stuffing the full original into every processing step, agent loops carrying every past observation long after it stopped being useful. The cure is the question of what this step actually needs to know, answered honestly, with everything else cut.
The second is an agent loop with no reliable stop. A loop that may take another step usually will, because from inside the model more work looks like thoroughness, and the model is never the one watching the bill. The fix lives in code rather than in the prompt: a hard maximum step count, a log line for every step, and where possible a stop condition checkable from outside the model. An agent loop without a hard limit is a billing incident that has not happened yet, and I no longer ship them.
So the habits, in one place: match the model tier to the task, trim context hard, cache the stable prefix, batch the patient work, go local only when volume justifies it, cap the spend, and read the dashboard weekly. None of it is exotic. The point was never spending as little as possible; the point is knowing where the money goes well enough that every dollar is chosen instead of discovered.
More ground level notes on running AI in production are at [xavierfok.com](/).
Get new guides and videos first — join the Telegram channel.