Running AI locally versus paying for APIs, a year into both
# Running AI locally versus paying for APIs, a year into both
One piece of advice follows me around every corner of the internet: run your models locally and stop paying API bills. On the surface the logic holds. Owning beats renting, surely. I have spent the past year running AI both ways for work that ships daily, some of it on a graphics card in my own machine, some of it through metered cloud endpoints. The honest scorecard is messier than any thumbnail admits.
What each path actually is
Local means the model weights live on your computer and your own hardware does the inference, usually a graphics card. You download an open model, you run it, and nothing you feed it leaves the machine. The API path puts the model in someone else's data center. Your request travels over the internet, their hardware does the work, and you pay per use.
Same goal, different shape. One is property you maintain. The other is a service you rent and rarely think about. Nearly every serious setup I know of, mine included, ends up using both at once, and choosing deliberately per task beats picking a side.
Where local genuinely earns it
Privacy comes first. When inference happens on your own machine, sensitive material never leaves it. Client work, unpublished drafts, private notes: none of it gets logged on a server outside your control.
Marginal cost comes second. Once the hardware exists, one more job costs almost nothing. I generate images and render video locally, and the thousandth render costs what the first one did, which is a sliver of electricity. At high volume that rewrites the economics entirely.
Then there is independence. No provider can throttle my pipeline mid afternoon, reprice it overnight, or retire the exact model a workflow was built around. And the whole thing runs offline. A bad night at some company's data center changes nothing on my desk.
What the cloud quietly wins
The savings crowd skips this half. Raw capability sits at the top. The strongest hosted models are simply larger and sharper than anything that fits on a consumer card, and on genuinely hard reasoning the gap is unmistakable. When only the best answer will do, honesty points at the cloud.
Speed is next. A modern data center accelerator finishes in seconds what my older card chews on for minutes, and while iterating, that difference changes how the work feels. Upkeep rounds to zero as well: no drivers, no memory juggling, no model refusing to load at midnight. A request goes out and an answer comes back.
The one everybody forgets is scale on demand. A hundred jobs at once just get absorbed in parallel. My single machine would queue them and grind for an hour. Part of the API price is the privilege of never thinking about any of this, and for plenty of people that alone is worth the money.
The bill nobody puts in a thumbnail
The it is basically free framing of local AI leaves out the entire catch.
Hardware leads. A genuinely capable card costs hundreds to thousands of dollars up front, spent whether it then runs one job or ten thousand. Electricity follows, small per job and constant in aggregate.
Time dwarfs both. Every hour spent fighting a driver, coaxing an oversized model into memory, or tracing a render that came out silently corrupted is an hour of making nothing. The API charges a little cash and almost nothing else. Local charges some cash and then a steady drip of attention forever, and attention runs out long before money does. That line never appears on any bill, which is exactly why people budget for everything except it.
My numbers, held loosely
I have not tracked every cent, and your numbers will differ, so treat this as the shape rather than gospel.
The hosted model I use for drafting and tightening text costs a few dollars a month at my writing volume. The metered voice service lands well under a dollar per full narration. Month after month those figures stay small enough to surprise me.
The local side looks free on any monthly statement, and that is an illusion. A graphics card is quietly amortizing, power is being drawn, and dozens of hours of setup and maintenance are spread across the year. Added up fairly: the cloud costs me real but trivial money, and the local setup costs me little cash and a serious chunk of time. Two different currencies. The question is which one you hold more of.
The rule I keep landing on
Choose per task. The cloud earns the job when the task is occasional, when it needs the smartest available model, or when speed matters more than control. That covers one off work, anything where the quality of a single answer is the whole point, and anything done only now and then.
Local earns the job when the volume is high, when the data is genuinely sensitive, or when the same task will repeat thousands of times and metered pricing would quietly stack up. Bulk processing, private material, background loops that never stop.
Notice there is no team to join. Almost every working setup, mine included, runs both at once without any tension.
Where my own split falls
The heavy daily work, rendering video and generating images, stays local on my own card, because it repeats constantly and metered pricing would pile up at a frightening rate. The thinking work, drafting and tightening writing, goes to a hosted model, because I want the best available quality and I only call it a handful of times per video. The voice sits in the middle on a metered service: the cost is tiny, and running genuinely good speech locally is more trouble than it is worth to me right now.
No flag got planted anywhere. Each job just went where it belonged. Anyone insisting the answer is always local or always cloud is describing a clean story rather than a working setup, and working setups are rarely clean.
The mistake pattern I keep seeing
People treat this as a money decision when it is a time and risk decision wearing a money costume. Someone buys an expensive card to save a few dollars a month in API fees, loses whole weekends to drivers and maintenance, and files the result under winning. The mirror image is worse: pushing genuinely private data to a cloud service to dodge a one time hardware cost, without once asking where that data ends up.
Three questions actually decide it. How much is your time worth to you. How sensitive is the data, truly. How much do you want to own rather than rent. Answer those straight and the choice mostly makes itself.
The line between them keeps moving
Open models you can run at home keep improving at a pace that genuinely complicates the picture, in a good way. The distance between a model on your own card and the big hosted services is narrower than it was a year ago. For everyday work like summarizing or classifying or drafting something simple, a local model is now completely adequate, and picking it out in a blind test would take effort.
The frontier is a different matter. The sharpest reasoning on the hardest problems still lives in the cloud, and the hardware it takes to serve those models suggests that will hold for a while. So neither side has won. The border shifts every few months, and a periodic review of your own split is the sober response.
If you are starting from zero
Do not buy a graphics card on day one. Start with the APIs, which cost close to nothing at small volume, and let the actual work reveal what you need. Build the thing first. Watch which task you end up running over and over, because that repeated, high volume task is the only candidate worth moving local later.
Most people run this exactly backwards: an expensive card first, bought on the vague feeling that owning must be cheaper, then a search for something to run on it. Hardware should answer a problem you have already measured. When the same job runs hundreds of times a week and the meter keeps ticking, that is the signal. Before then, it is a guess.
Local is a tool that bills you in time. The cloud is a tool that bills you in cash. Put the repeated work on hardware you own, rent the occasional work, and stay honest about which is which. More of these breakdowns, with the real tradeoffs intact, live at [xavierfok.com](/).
Get new guides and videos first — join the Telegram channel.