How AI tokens actually work
# How AI tokens actually work
There is one word that, once you properly understand it, explains most of what confuses people about working with AI. It explains what you get charged. It explains why a chat suddenly forgets something you said ten minutes earlier. It explains the hard ceiling on how much text you can paste in at once. The word is token, and most people skate past it on a bill or in a settings menu without ever stopping.
I use these tools all day for real work, and this single concept has saved me more money and more frustration than any prompt trick. So here is the plain English version, no math required.
A token is a chunk, and usually a fraction of a word
When you type a sentence into an AI, the model does not see words the way you do. It does not even really see letters. Before anything happens, your text gets chopped into small pieces called tokens.
Here is the part that surprises people: a token is usually a piece of a word rather than a whole one. A common short word like "the" or "cat" is a single token. A longer or rarer word gets split. "Unbelievable" might arrive as un, believe, and able. Spaces and punctuation count as well, so a sentence you think of as ten words can easily be thirteen or fourteen tokens once the chopping is done.
The rough rule of thumb: one token is about three quarters of a word, which means a hundred words comes out near a hundred and thirty tokens. It is approximate, and approximate is fine. The point is that the model reads in chunks, and those chunks are the unit everything else is built on.
This already clears up a few small mysteries. The same idea can cost slightly different amounts depending on phrasing. A wall of code or a table of numbers consumes far more than its visual length suggests, because all those symbols and odd fragments shatter into many tiny tokens. The model counts chunks, and chunks refuse to line up neatly with anything you would naturally count yourself.
Why chunks beat whole words
The obvious question is why anyone designed it this way. Why chop language into odd fragments instead of using words like a normal person?
The answer is genuinely elegant. There are millions of words across the world's languages, plus names, typos, invented terms, and every niche piece of jargon. No model could hold a separate slot for each one; the list would be impossibly large. Break language into common chunks instead, and a vocabulary of a few tens of thousands of pieces can build absolutely anything. The model can spell out a word it has never seen, piece by piece, the way you would sound out an unfamiliar name.
Consider the alternatives and the design gets clearer. Whole words would demand an endless list and still choke on anything new. Single letters would keep the list tiny while forcing the model to crawl through everything one character at a time. Tokens sit in the sweet spot: common words stay whole, so they are fast and cheap, while rare words break into pieces, so nothing is impossible to represent. It works like a sensible filing system, where common items get their own drawer and rare items share a general one, and between them the cabinet stores everything without being infinite.
The model writes in tokens too
Tokens are the model's output format as much as its input format, and this is the piece to really hold onto. When an AI answers you, it produces the reply one token at a time, predicting the next most likely chunk, then the next, all the way to the end. That is the entire operation under the hood. It is why a streaming reply appears in little bursts rather than all at once: each burst is another handful of tokens being produced.
So every interaction carries two counts that matter. Tokens in, which is everything you sent. Tokens out, which is everything it said back. Both numbers drive what you pay, and both fill the limited working space we get to next.
The context window is a desk, and there is no filing cabinet behind it
This is the idea that dissolves the most confusion. A model can only pay attention to a limited amount of text at any moment. That space is called the context window, and it is measured in tokens.
Picture a desk. Everything the model is currently working with has to sit on that desk at the same time: your question, the conversation so far, any document you pasted. The desk is only so big. A fresh chat starts with the desk empty and roomy, and every message in either direction lands on it and takes up space.
Now the part people resist. The model has no memory in the human sense. It does not recall your last conversation, and within a chat, the only things it knows are whatever currently sits on the desk. Each time you send a message, the model walks up and reads the whole desk from the top, as if for the first time. When it appears to remember your name three messages later, that is only because your name is still physically present on the desk. Once something falls off, the model has no way to know it ever existed.
The bill is just the desk, metered
With the desk in mind, pricing stops being mysterious. Using these tools through their raw interfaces, you pay per token, in both directions. A short, tight question costs almost nothing. Pasting a giant document and requesting a long detailed essay costs noticeably more, because you loaded the desk heavily and asked for a heavy reply.
The subtle one is conversation length. Every new message in an ongoing chat quietly drags the entire history with it, because the full desk gets re-read on every single turn. A long chat is a cost that grows as it goes. None of this is a hidden gotcha. It is a very literal meter running on the amount of text in and out, counted in chunks.
Why long chats go foggy
The desk also explains the most maddening behavior these tools have. Deep into a long conversation, the AI forgets something you clearly established near the start. It contradicts itself, or asks for a detail you already provided. People take it personally.
What actually happened is that the desk filled up. When a conversation outgrows the context window, the oldest material slides off to make room for the new. Whatever slid off is gone entirely, as though it was never said. The forgetting is physical, so to speak, rather than careless. The same mechanism is why very long chats gradually turn slower and pricier: the desk is fuller on every turn, and re-reading a fuller desk takes more work.
The rules that fall out on their own
Turned into practice, the picture yields a short list. Be concise, and paste only what the model truly needs, because every extra page on the desk costs money and attention. When a long chat starts forgetting, stop fighting it: open a fresh conversation and carry over just the key points, which hands the model a clean, roomy desk. When something seems expensive, remember that the pasted document counts, the long answer counts, and in an ongoing chat the whole history counts again on every turn. And a paste limit is simply the size of the desk, with no workaround beyond putting less on it.
What I like about this list is that it does not need memorizing. Of course the long document costs more; it loads the desk. Of course the chat forgot; the desk overflowed. Of course a fresh conversation fixes it; you cleared the desk. Once the picture is in your head, every tip becomes something you can re-derive on the spot.
The fuzzy edges, stated honestly
A few caveats, so nobody treats my numbers as gospel. Exactly how text gets chopped varies between models, so the same sentence can tokenize to slightly different counts on different systems. The three-quarters-of-a-word rule is rough, and it gets less reliable for code, for languages other than English, and for unusual text. Desk sizes differ per model and keep growing as the technology improves, which is straightforwardly good news.
The shape of the idea is the durable part. Text becomes chunks, chunks fill a limited desk, and the meter counts chunks. The specifics drift around that core, and the core has held for as long as I have worked with these tools.
One word, most of the mystery
Of everything you could learn about how AI works underneath, I think this is the highest value single idea, which is why it deserved a full write-up on its own. Understand tokens and your bill stops surprising you. Long chats going foggy stops being an annoyance and becomes a known mechanism with a known fix. You start writing tighter, cheaper requests almost without thinking about it. Ten minutes of genuine understanding, traded for never having to look any of this up again. I keep publishing plain-language breakdowns like this one at [xavierfok.com](/).
Get new guides and videos first — join the Telegram channel.