The self hosted AI assistant I keep running around the clock
# The self hosted AI assistant I keep running around the clock
Most AI assistant setups share a quiet flaw: they exist only while you are looking at them. You open a tab and talk to a model, you close the tab, and whatever it was doing is gone. Nothing carries over and nothing waits for you. What you have is a calculator with better manners.
I accepted that for a while, because that was simply how these things worked. Eventually I got tired of it and built the alternative: an assistant that runs on my own hardware around the clock, kicks off work on its own schedule, and has an answer sitting there when I come looking. This is the honest account of what that looks like, including the weekend it silently died.
What self hosted means here
The phrase gets used loosely, so here is the full definition I work from. The model runs on hardware I own and control, in my room. The data I feed it stays on the machine unless I deliberately send it somewhere. The process runs whether or not any browser tab is open. And where the setup does cost money, the cost is metered and under my control, never a subscription seat priced by someone else's decisions.
None of this is a stance against cloud services. I use them constantly. The point is knowing exactly where processing happens, which starts to matter a great deal once a system is reading your own notes and files and watching your schedule.
Why I bothered
Rate limits pushed me first. When everything ran through hosted API calls, busy days kept hitting the ceiling, and a self hosted model has no ceiling at all. A hundred calls in a row throttle nothing.
Cost and privacy did the rest. A local model has near zero marginal cost per call; the electricity is rounding error. That changes the ledger completely for routine, low stakes jobs that fire dozens of times a day. And some of what I wanted the assistant reading is my own notes, drafts, and half-finished files, which I never liked routing through someone else's API on every call. Keeping that processing in the room removed the question entirely. Those reasons together made the effort worth it, and none of them is ideological.
The shape of the thing
I will describe the architecture without naming a stack, because the shape matters more than the tools. At the center is a small language model running locally. It handles short bounded tasks well: summarise this, classify that, reformat a list, answer a question about one document. It makes no attempt to be a frontier model. Its job is to be always available and dependable for narrow work.
Around the model sits a layer of scripts, each with exactly one job: call the model with some context, take the output, do something with it. A scheduler fires those scripts on a timetable. A chat surface makes the whole thing reachable from my phone or desktop. Model, scripts, scheduler, chat surface. That is the entire machine.
A normal day
The morning digest comes first. Overnight, jobs either completed or failed, and at breakfast the assistant sends a short plain language summary: what changed overnight, which jobs finished, which ones broke, what the calendar holds. No searching and no deciding, just collecting the state of things and presenting it.
During the day it answers questions about my own notes and files. Asking what I wrote about a topic three months ago and getting it found and summarised in seconds is something I could not do before, and I use it more than I expected to.
It also triggers the scheduled jobs: render this, move that, send the report, tidy the folder. The scripts do the actual work; the assistant starts them at the right time with the right arguments. And it flags anomalies, a failed job or a log file growing past a threshold. That is the whole daily loop. Nothing in it is magic, and all of it is useful.
Where a small model stops
The limits deserve plain statement, because they are real. A small local model reasons more slowly than a large frontier model, makes more mistakes on complex work, and holds far less context at once. For the bounded jobs above, none of that matters much. For long form reasoning, for writing that has to carry tone, or for anything that needs a lot of context held in one head, it is simply outmatched, and pretending otherwise produces bad output on a reliable schedule.
Routing the hard questions out
So the working architecture is a split, and it took me a while to land on it. When a request comes in, the script checks a few signals. Is the task short and bounded, or does it sprawl. Does the context fit the local model comfortably. Would a rough answer be acceptable here. Or does this need to be exactly right.
Any of those pointing at the hard bucket sends the request out to a paid frontier model: one metered call, a much better answer, and the result logged and delivered into the same chat surface as everything else. From my side the seam is invisible, because the switching happens in the script rather than in my head. The metered bill stays small since only the genuinely hard requests leave the building. The local model carries the volume and the paid API carries the difficulty, and that division of labor is what makes the whole thing practical.
Keeping it alive is the actual work
Getting the assistant running the first time took maybe a day. Keeping it running reliably for weeks on end is a different job entirely, and nobody talks about it enough.
The model process crashes occasionally. The scheduler sometimes misses a job. A script that worked last week breaks because something upstream changed shape. The disk fills up if nobody watches the logs. None of these are dramatic failures; they are the ordinary entropy of a system with moving parts. Build for them, or one morning you come back and find nothing has run for two days and you have no idea why.
I learned this the honest way. The first version had no reliability layer at all, and after one weekend away I found it had been sitting idle the entire time.
Boring reliability, which is the good kind
Every script writes a short log line when it runs: what ran, whether it worked, how long it took, what it returned. Once a day a health check script reads those logs and reports. It flags missed runs and repeated errors, and it pings the model process to make sure it still answers. The logs are plain text files, and the health check is maybe fifty lines of script, built in an afternoon and then forgotten because it quietly does its job without asking for anything.
Restarts are the other half. A supervisor process brings the model back up when it dies, and a scheduled job that fails three times in a row alerts me instead of silently giving up. Process supervision is decades old; the only mistake is assuming a personal system is too small to need it. Everything that runs around the clock needs a watchdog. Wiring one up takes half an hour and buys you out of waking to a dead assistant two weeks after launch. Once the restart policy exists, you stop thinking about whether the process is up and start thinking about what it is doing, which is a far better problem to have.
What I would tell someone starting
Start with one job. A general assistant that does everything is the wrong first build. Pick one specific thing you actually need done on a schedule, a morning digest or a daily backup check, make it dependable, and let it run for a few weeks before adding anything else. The hard part is trust rather than code. You need to watch a system fail for a while before you know it well enough to hand it more responsibility, and one reliable job that restarts itself is worth more than ten flaky ones.
Build the logging in from day one as well. I told myself I would add it later, once things were working, and when something broke in week two I had nowhere to look. Rebuilding observability after the fact took longer than doing it properly at the start would have. Now every new script logs before it is ever tested, and that visibility is the difference between a system you trust and one you are vaguely worried about in the back of your head.
The result, honestly
The morning digest alone saves a noticeable slice of every day, because knowing the state of things without checking each system by hand turns out to be worth a lot. The file search gets used constantly. The routing split keeps the paid bill small. And the reliability work means I mostly forget the assistant exists, which is precisely what you want from infrastructure. It asks for no attention. It runs, and the answer is waiting when I come looking.
None of it is magic: a small model, a pile of scripts, a scheduler, and a healthy amount of boring reliability work. The payoff is real, and the stack is within reach of anyone willing to start small and grow it slowly.
Get new guides and videos first — join the Telegram channel.