The AI advice everyone repeats that I think is wrong
# The AI advice everyone repeats that I think is wrong
The prompt that tags every video I publish is about nine months old. I have not edited a word of it, it has survived two model swaps, and it is badly written. I wrote it in four minutes while annoyed, after burning most of a week on a different prompt that never worked.
That gap is where most of my opinions about AI come from. I run mobile proxy lines on carrier SIMs, a rack of Android phones rented by the week, and content sites with a video pipeline underneath. AI does the dull middle of it, and none of what follows comes from a paper.
Prompt engineering is a spec writing job with a bad name
Telling someone to learn prompt engineering as a career sends them at the wrong skill.
Here is the week I lost. One prompt was doing four jobs: read a transcript, work out the subject, choose tags from a fixed list, rate its own confidence. Tags came back fine, confidence numbers were garbage. I rewrote the wording for days. Polite version, terse version, the version where you tell the model it is a world expert.
Then I split it into three calls, each with one job and an output a script can check. The wording in all three is clumsy. It worked that afternoon.
What fixed it was cutting the problem into pieces with checkable answers. That is a systems skill, it is forty years old, and nobody sells a course in it under that name.
Which is why the old tagging prompt still runs. It asks for one thing and states the shape the answer must arrive in, so there is nothing clever in it for a new model to break. Every prompt I polished is gone, because vendors fixed the behaviour I was writing around and took my phrasing with it.
Long unsupervised chains are oversold
The useful shape is a short loop with a human gate on anything you cannot take back.
Start with the arithmetic. At ninety five percent per step, a ten step chain lands correct about six times in ten. Reality is worse, because the errors compound. Step four builds on step three's wrong answer and produces something confident and broken.
A crash is the good outcome. A crash stops. What actually happens is that the run completes, every step logs success, and the damage sits there looking normal.
So my gate is drawn on reversibility. A customer port here is pinned in the database so that no automation may swap it to a different modem, whatever the health checks think. The swap logic is good. It is also allowed to be wrong about a paying customer exactly zero times.
Deleting, sending, charging, publishing, messaging a customer, cancelling a line: a person looks. Reading, drafting, tagging, summarising, proposing: unattended, checked afterwards. The rule is two years old and has cost me nothing.
The demos are real. They are also always the same handful of tasks, because those have a clean success signal at the end. Book the flight or do not. Most of my work fails quietly and surfaces two weeks later.
Most local model requests are about discomfort
I say this with a GPU in the room doing real work daily.
Local buys a specific list. No rate limits, so an overnight job can make forty thousand calls. It works when the connection does not. No vendor inside a pipeline I wrote two years ago. And for a couple of things I handle, data that never leaves the building.
What it does not buy is a lower bill. I stopped claiming that a year ago.
The question I ask now is: name the document. Customer identity papers, a file covered by an agreement you signed, a medical record. If you can name it, local is correct and you should already be there. If you cannot, what you have is a feeling, and a feeling is not a compliance requirement.
Meanwhile the same person pastes that text into an unaudited browser extension and a spreadsheet whose terms they never opened. The API call is the only link in that chain with a written contract attached. The paid tiers of the large providers state whether they train on your traffic and how long they keep it. Reading yours takes less time than downloading a model.
And local is not private by default. A weights file off a hub with unclear provenance, an inference server listening on every interface with no password, on a box running five other jobs. I have had that by accident, and it feels safer than the API call it replaced, which is the dangerous part.
The model is almost never the problem
Upgrading beats fixing the plumbing far less often than people assume.
Two failures from my own logs. A scheduled job and the app it belonged to read different environment files, so mail went out through the wrong server for weeks with nothing complaining. And a database client reused one connection across threads, so a thread got another thread's answer. That second one is the worst class of bug I know, because the data is real and belongs to somebody else.
Neither has anything to do with model quality. Both cost me more than the gap between any two frontier models today.
When a model I depended on was retired, the swap took an afternoon, because the call site was one function in one file. The pipeline around that function took two years. That ratio is the argument.
A new model also arrives with new quirks: different refusal behaviour, different tolerance for your output format, a different price. All of it lands on the plumbing anyway, so the free upgrade costs a week of small repairs.
When my output is bad, the model is the last thing I check. It ranks between fifth and never.
AI writing passes a glance and fails a read
This one costs people money.
The failure mode is specific: a wrong sentence and a right sentence look identical. Same rhythm, same easy confidence. Skimming catches typos and is useless against fluent invention, so the cheap check does not exist.
I have timed it. Drafting two thousand words with a model takes about twenty minutes. Verifying them properly takes longer than writing them by hand, because checking someone else's confident sentence means establishing whether a claim you would never have made is true. That is a research task you did not choose.
I published a figure once that came out of a draft and was never checked. One number, one line, everything else correct. It did not help.
Where it does work is anywhere a machine can do the checking. Tags, because a script confirms the tag exists. Classifications, because I keep a file of known answers. Code, because it runs or it does not. A first pass I was going to rewrite anyway. The common error is taking the tagging job's economics and applying them to the essay.
Leaderboards do not describe your workload
My jobs look nothing like a benchmark. Return valid JSON with these five fields, twenty times out of twenty. Read a transcript and pick the three moments worth putting on screen. Say you do not know instead of guessing.
That last behaviour is the most valuable one in a pipeline that runs while I sleep, and I have never seen it scored.
A real trade: the model doing my tagging does not write the best tags of the candidates I tested. It wins on returning a parseable answer every single time. A smarter model that prepends a friendly sentence to the JSON one run in twenty costs me a retry path, a parser that tolerates rubbish, and a new category of silent failure. A two point ranking gap appears nowhere in that.
What I use instead took one afternoon. Twenty rows whose answers I know, in a file. Run each candidate, count hits, then read the misses, because how a model fails tells you whether you can guard against it. That file has decided every model choice since, and it has disagreed with the rankings more than once.
Where I am probably wrong
The agent position is the one I trust least. I built those loops two years ago around models that could not hold a long chain, and I have not seriously retested since. I may be defending a design decision and calling it a principle, and because the pipeline works, I would be the last to notice.
The writing position assumes my standard of correct. My name is on these sites. Someone writing internal notes, where eighty percent right and five times faster is a good trade, has different arithmetic and is right to take it.
The local position is argued from a country with cheap power and a connection that stays up.
And the leaderboard position has an obvious hole: I only test three or four candidates, and I pick them using the rankings I just told you to ignore.
What I changed my mind about
I used to insist a person reads everything a machine writes. No exceptions. I argued it as a standard.
It was superstition in one specific place. Where a script can verify the output, a human reading it too adds delay and nothing else. My tag checker catches things I miss on the fortieth item at eleven at night, and boredom is the real failure mode of a person doing quality control.
So the question is whether the thing can be checked without me. Whether a machine produced it is beside the point. That took a year and a growing backlog to admit, because the old rule felt like integrity and was partly habit.
The other half has not moved. Anything where the check requires knowing whether a claim about the world is true still comes to me, and I cannot see that changing on any timeline I would bet on.
Get new guides and videos first — join the Telegram channel.