Building an AI agent with tools: how function calling actually works
# Building an AI agent with tools: how function calling actually works
A language model on its own has no hands. It can describe how it would rename your files, draft the search query it would run, explain the email it would send. None of that touches anything. The model lives inside a conversation, produces text, and stops there.
For plenty of tasks that is fine. The wall appears the moment you want the model to act on the world instead of describing it. The idea that gets you over that wall is called function calling, and the name has confused nearly everyone I have explained it to, because it suggests the model executes code. It never does. Once you see what actually happens, agents stop looking like magic and start looking like something you could build in an afternoon.
Which you can.
The mechanism, minus the misleading name
You begin by describing tools to the model in plain language. A tool here is just a function your own code knows how to run: read a file, query a database, check a calendar. For each one you give the model a name, a short description of what it does, and the inputs it expects.
The model holds those descriptions in mind during the conversation. When it decides a tool would help, it replies with a structured message instead of an answer. The message says, in effect, please run this tool with these inputs. Your code receives that request, runs the real function, and feeds the result back into the conversation as the next message. The model reads the result and carries on.
That is the entire mechanism. The model requests. Your code executes. Every piece of actual work happens on your side of the line, which means every safety decision lives on your side too. The model can ask for whatever it likes, and your program decides what actually runs. All the trust you will ever place in an agent rests on that boundary.
Why one small idea unlocks so much
Anything you can wrap in a function, you can hand to the model. A search engine. A database. A file system. A calculator. The model needs no understanding of how any of it works internally. A clear description of what goes in and what comes out is enough.
The part that turns this into an agent is the loop. The model calls a tool, reads the result, decides it wants another call, reads that result too, and keeps going until it has enough to answer. One question from a user can trigger five tool calls behind the scenes, none of which the user ever sees. From the outside it looks like initiative. From the inside it is a while loop that runs until the model stops requesting tools.
A real agent in two functions
The smallest agent I would call real answers questions about your own files. It needs two tools. One lists the files in a folder. One reads a named file and returns its contents.
A user asks a question. The model sees the question plus the two tool descriptions, decides it should find out what files exist, and requests the listing tool. Your code returns the list. The model scans it, picks a file that looks relevant, and requests the read tool with that filename. Your code returns the text. Now the model has what it needs and writes the answer.
Nothing in that sequence is complicated, and it is still a genuine agent. It chose its own steps, gathered information nobody handed it directly, and produced an answer it could never have produced alone.
Two functions and a loop.
Tool design decides how smart the agent looks
The quality of an agent tracks the quality of its tools far more closely than people expect. The rule that matters most: keep every tool narrow. One tool, one job.
A single tool called search that hits your files or the web or a database depending on a mode flag will produce bad decisions, because the model has to guess which mode fits the moment. Split it three ways instead, one tool for files, one for the web, one for the database, and the guessing disappears. The model reads descriptions to make its choices, so write them like instructions rather than labels. I have watched an agent go from clumsy to reliable on the strength of rewritten descriptions alone.
Output deserves the same care. The model has to reason about whatever a tool returns, and a dump of raw rows with cryptic column names will sink it. Treat every return value as a message to a reader. Label things plainly. The clearer the result, the better the next decision.
The two guardrails I refuse to skip
Reading is safe. Writing, deleting, sending, publishing: those carry real risk, because the model triggers them from a natural language request it may have misread.
The first guardrail is a confirmation step in front of anything irreversible. When the model requests a destructive call, my code holds the request and surfaces it to me instead: the agent wants to delete these three records, yes or no. The tool runs only on yes. The friction is small, and it is precisely what lets me trust the system with sharper tools over time.
The second is logging every call. Tool name, inputs, output, timestamp, written to a plain file beside the agent. It costs a few lines of code on day one and has saved me hours of guessing when a result came out wrong. An agent that ran quietly and left no trace can never be diagnosed. You cannot build trust in a system you cannot inspect.
Where agents earn their keep
Boring multi step chores. That is the honest answer, and I mean it as praise.
Pulling data from three places and merging it into one report. Working through a batch of files and renaming each one based on what is inside. Applying the same check to every item on a list and flagging the failures. These are tasks where the procedure is known, each step has a checkable result, and the only reason a human finds them slow is the sheer number of small moves. Agents follow procedure well. The loop runs, the tools fire in a sensible order, and the output is easy to verify at the end.
Where they fall apart
Open ended goals wreck them. Hand an agent a task with no clear stopping condition and it will spin, wander down wrong turns, or declare itself done at a point that is visibly unfinished. Judgment that depends on context outside the conversation fails just as reliably. An agent tuning a database query can tune it correctly and still break a downstream report it had no way of knowing existed.
The limitation is visibility. An agent sees the conversation and the tool results, and nothing else. It has no history with your colleagues, no map of what else touches the system, no sense of consequences beyond the task in front of it. Narrow and well defined goes well. Broad and fuzzy goes badly. That is the shape of the tool, and no clever prompt reshapes it.
When to skip the agent entirely
If you already know the exact steps, write ordinary code. An agent earns its complexity only when the path cannot be specified in advance, when each next step depends on what the previous one found. A fixed sequence wrapped in an agent is a slower, flakier script.
The same logic covers plain text work. A single well prompted call with no tools at all beats a sloppy agent at anything that amounts to producing good writing. The value of an agent is adaptability. If the task has no variability, the agent has no job.
Start smaller than your ambition wants
Two tools and a tight loop. That is where I would point anyone building a first agent, because it is where nearly every useful agent I run actually started.
Pick two functions your code can already perform. Keep the task narrow and the definition of done unmistakable. Wire in logging before anything else. Then run it, watch it, and read what it did call by call until the model's decision patterns stop surprising you. Ten tools and a grand goal on day one produce a system that behaves in ways you did not expect and resists debugging when it misbehaves. The small version teaches you how the model chooses, how descriptions steer it, and where the loop holds or slips. Growing it from there is the easy part.
There is plenty beyond this foundation: tool schemas, error handling, memory that outlives the context window, agents that call other agents. All of it stands on the same base. A model that can request actions, code that executes them, tools kept narrow and safe, and the habit of reading what actually happened.
If you want to watch me build and run systems like this on my own infrastructure, with the failures left in, there is more at [xavierfok.com](/).
Get new guides and videos first — join the Telegram channel.