Xavier Fok
← All writing

· 5 min read

Teaching an agent to say it does not know

Why agent uncertainty is hard to build, and how grounding, tool checks, and escalation paths get a production agent to admit when it's guessing.

The default behavior is to bluff

Run enough agents against real tasks and you’ll notice the same failure pattern over and over: the agent doesn’t fail loudly, it fails confidently. Ask it something outside its knowledge and it doesn’t pause. It picks the most statistically plausible answer and states it like fact, formatting and all. That’s not a bug specific to one model or one vendor. It’s what happens when a system is trained to predict the next likely token, not to check whether that token is true.

I run agents daily for content, video pipelines, and infrastructure work, and the single most useful thing I’ve done to any of them isn’t a smarter prompt. It’s building an actual path for “I don’t know” to exist as an output, separate from every other answer. That sounds simple. It isn’t, because nothing in how these models work gives you that path for free.

Why “just ask it to rate its confidence” doesn’t work

The obvious first move is to add a line to the system prompt: “state your confidence level” or “say if you’re unsure.” It helps a little, but it’s shallow. A language model doesn’t have introspective access to a calibrated confidence score the way a classifier does. When you ask it how confident it is, you get another generated answer, produced with the same next-token guessing process that produced the original claim. It can say “I’m 90% confident” about something completely wrong, because that phrase was statistically plausible to write next, not because it ran an internal check against a ground truth.

Self-rated confidence is worth having as a soft signal, especially paired with something like asking the model to answer the same question a couple of different ways and seeing if it stays consistent. Disagreement across attempts is a real, if noisy, uncertainty signal. But treating a stated confidence number as reliable on its own is how you end up with an agent that’s wrong and sure about it, which is worse than an agent that’s wrong and hesitant.

Uncertainty has to live in the system, not the prompt

The fix that actually holds up in production isn’t asking the model to be more honest. It’s changing what the model is allowed to answer from. If an agent only answers from things it can verify, “I don’t know” becomes a natural, structural outcome instead of a personality trait you’re begging for.

Concretely, that means grounding answers in retrieval or tool calls instead of parametric memory whenever the question has a checkable answer. If an agent is answering “what’s the status of this order” or “what does this config value do,” it should be calling a database or reading a file, not recalling something that resembles the answer from training. When the retrieval or tool call comes back empty, that’s your uncertainty signal, and it’s a real one, not a generated guess about a guess.

Let tools return “not found” instead of forcing an answer

This is the part people skip. When you wire an agent into MCP servers or any other tool layer, it’s tempting to make every function return something usable, because an error looks like a broken integration. But a tool that always returns a plausible-looking result, even a partial or synthetic one, removes the one clean signal the agent had for “this doesn’t exist.” A search function that returns the closest match instead of nothing when there’s no real match is training the agent, in that session, to treat approximate as exact.

I set my tools to return an explicit null or empty result when there’s genuinely nothing there, and I write the agent’s instructions to treat that null as a stop condition, not something to paper over with a best guess. It’s a small design choice at the tool layer, but it does more for honest behavior than any amount of prompt tuning, because the agent isn’t being asked to police its own certainty. It’s being handed a fact: nothing came back.

Give the “I don’t know” path somewhere to go

An agent that says “I don’t know” and then stops is only half useful, and in a fully autonomous pipeline it can be actively unhelpful if nothing catches that output. The path matters as much as the admission. In my own pipelines, when an agent hits a genuine gap, whether that’s a missing data point, a tool returning nothing, or low agreement across repeated attempts, the run doesn’t silently continue with a guess. It logs the gap and either stops that item or routes it to a queue a person checks later.

That’s a deliberate tradeoff. It means the pipeline is slower and less “hands off” than it would be if the agent just filled in the blank and moved on. But a blog draft, a video script, or an automated reply that quietly fabricates a fact because the agent didn’t want to leave a blank costs more time to catch after the fact than the pause costs upfront. Building the escalation step is what makes “say you don’t know” a workable instruction instead of a nice sentence in a system prompt that gets overridden the moment the model finds any plausible-sounding continuation.

Local models make this harder, not easier

Running smaller models locally instead of hitting a large cloud model is often the right call for cost, latency, or keeping data in house, and I do it regularly for parts of my pipeline. But it’s worth being honest that smaller local models tend to be worse at flagging their own uncertainty, not better. They still guess when they don’t know, and their self-reported confidence is noisier on top of that. If anything, running local pushes more of the uncertainty-handling work onto your system design: stronger grounding, more tool checks, tighter escalation, because you can’t lean as much on the model itself to hedge appropriately. That’s not a reason to avoid local models. It’s a reason not to relax the guardrails just because the workload moved off a bigger cloud model.

What this buys you and what it doesn’t

None of this makes an agent smarter or gives it real self-awareness. It’s plumbing: grounded answers where grounding is possible, tools that admit when they’ve got nothing, and a place for genuine gaps to land instead of getting silently filled in. That’s a modest, mechanical thing, not a leap toward a system that understands its own limits the way a person does. But it’s the difference between an agent you can actually leave running against real tasks and one you have to double check line by line, because you never know which confident-sounding sentence is grounded and which one is a guess wearing the same font.

If you’re wiring agents into real workflows, that plumbing is most of the job. The prompt is the easy five percent.

If you want more on running agents and AI tooling in actual production setups, not demos, head back to the homepage for the rest of what’s on the channel.

Get new guides and videos first — join the Telegram channel.

Keep reading