The work AI made worse

The parts nobody puts in the demo

Every AI product demo shows the same thing: a clean prompt, a clean output, a happy operator nodding along. What it never shows is the six months after, when that workflow is running unattended at scale and something quietly starts going wrong in a way that looks fine from the outside.

I run agents and local models against real production work every day: content generation, video rendering, task automation across a fleet of machines. Some of that work got faster and better. Some of it got worse in ways that took a while to even notice, because the failure mode wasn’t a crash. It was a slow degradation that passed every surface-level check.

This is a list of the specific ways that happened, and why, mechanically, they happened. Not a doom piece. Not a “AI is bad” piece. Just an honest accounting from someone who has to live with the output.

When “it ran successfully” stops meaning anything

The first category of harm is the exit code trap. A script calls an API, gets a 200 response, writes a file, and reports success. Nothing in that chain checks whether the file is actually complete or correct.

I hit this with text-to-speech generation feeding a video pipeline. The synthesis call would occasionally return a truncated audio stream, sometimes a third of the expected length, sometimes half. The script didn’t crash. It didn’t retry. It saved the partial file, saw that a file now existed at the expected path, and treated that as done. Downstream steps saw a file, not a duration, so they didn’t catch it either. A batch of videos went out with audio that cut off mid-sentence before anyone noticed, because “did it run” and “did it work” got treated as the same question.

This isn’t a flaw unique to TTS. It’s a flaw in how automation gets wired to AI services generally. The API contract is “I attempted the thing,” not “the thing is correct,” and if you don’t add your own verification layer, you inherit that gap. AI didn’t cause this problem. It made it cheaper to build pipelines with the gap left in, which means the gap shows up more often, at more places, before anyone’s had time to notice.

Tagging and classification: good enough to hide, not good enough to trust

A second pattern shows up whenever you use a model for classification or tagging inside a pipeline, rather than as the final output a human reviews. I’ve run auto-tagging on rendered content to route it into the next stage automatically. It works at roughly half the time. Just over half, honestly.

That number sounds like a coin flip you’d catch immediately. It’s worse than that, because a 50-55% success rate on an otherwise silent classification step means the failures don’t cluster somewhere obvious. They spread thin across everything, so no single batch looks broken. You have to go looking for the failure rather than tripping over it. I ended up turning off automatic refill from that tagging step entirely, because a human checking the tags took less total time than a human debugging why some fraction of content was misrouted for reasons that weren’t visible in any log.

The lesson isn’t “classification models are bad.” It’s that a model performing acceptably on a benchmark, or even acceptably on a spot check, tells you very little about whether it’s safe to remove a human from that specific loop. The cost of an occasional wrong tag when a person reviews the list is near zero. The cost of the same wrong tag rate feeding an unattended downstream process is a mess that took me longer to diagnose than the automation ever saved.

Agents that touch real infrastructure change what “reversible” means

The place I’ve had to be most deliberate is agent tooling wired into MCP servers that can actually do things: modify configs, restart services, touch routing rules on live servers. An agent that can read and suggest is low risk. An agent that can also act closes the loop, and closing that loop is where speed and blast radius trade against each other directly.

A concrete case: a routing rule on a server kept silently disappearing after service restarts, because the process that rebuilt the routing table on boot didn’t preserve a rule at priority zero. The fix was straightforward once found, pin the local rule at a higher priority number so it survives the rebuild. But an agent empowered to “fix broken servers” on its own, without that specific piece of tribal knowledge, would have looked at the same symptom and reasonably tried something else: restart it again, which is exactly the action that triggers the loss in the first place. Faster iteration on the wrong hypothesis is not the same as progress. It’s just a quicker way to make the same mistake more times.

This is the actual tradeoff with agents doing infrastructure work, and it’s not the tradeoff most people talk about. It’s not “will the agent understand my code.” It’s “does the agent know which of its own actions are the ones that made things worse last time,” and unless that knowledge is written down somewhere the agent reads before acting, it doesn’t. Every session starts flat. That’s a real cost of the speed, not a hypothetical one.

Where the false negative is worse than the true positive

There’s a specific shape of bug that AI made worse simply by being competent at surface-level tasks: a diagnostic script says “everything is fine” while checking the wrong signal.

I’ve seen this with server health checks that look at whether a management layer reports devices as online, rather than checking whether traffic can actually pass end to end. The management layer’s own view of connectivity is not the same thing as the customer’s experience of connectivity, and a script that only probes the former can report a clean bill of health for hours while the real path is dead. An AI-assisted monitoring setup is very good at building exactly that kind of check quickly, because it’s the check that’s easiest to describe and easiest to implement. It is not automatically good at identifying that the check is measuring the wrong layer, unless someone tells it, explicitly, which layer actually matters.

Fast, cheap monitoring that checks the wrong thing is worse than no monitoring, because it gives you false confidence instead of an honest gap. Before, you knew you had no visibility and acted accordingly. After, you believe you have visibility you don’t actually have.

What actually holds up

None of this means pull the automation out. The parts of my pipeline that got measurably better are still running: batch content generation, render queueing, most of the day-to-day glue work. The failures above share one trait, they all happened in unattended loops where the AI’s output fed directly into the next automated step with no human or hard verification step in between.

The fix in every case was the same, and it’s not exotic: check the actual outcome, not the reported outcome. Measure a file’s duration, not its existence. Sample the classification output, don’t just trust the aggregate score. Write down the specific failure mode after you find it, in a place the next run actually reads, instead of trusting the model to reconstruct that judgment fresh every time. AI is very good at doing the next step quickly. It is not good, by itself, at noticing that the step it just did was the wrong one, and that gap is exactly where cheap speed turns into expensive rework.

If you’re deciding where to trust AI in a real workflow, that’s the question worth asking before you flip anything on: not “is the model good enough,” but “what happens the one time in twenty it isn’t, and will anything notice.”

If you want more of this kind of unglamorous, specific detail on running AI in production, head back to the homepage for the rest of what’s here.

Get new guides and videos first — join the Telegram channel.