No jargon, promise
WTF are tokens?!
AI people say “tokens” like everyone was issued a glossary. Here is what they actually are, why context windows are measured in them, and when a normal person should care.
Checked against current official OpenAI, Anthropic, and Google documentation on August 30, 2026. Exact token counts, context limits, and accounting rules vary by model and provider.
First: tokens are not little AI coins
The word is doing us no favors. “Token” sounds like a tiny credit you spend to make the robot talk. That is not what it means.
Before a language model works with text, the text is split into smaller machine-friendly chunks. Those chunks are called tokens. The model processes tokens on the way in and generates tokens on the way out.
So when somebody says a prompt used 2,000 tokens, they are basically describing how much chunked-up information the model had to work through — not saying the user inserted 2,000 AI quarters.
Tokens are chunks of information a model processes. They are a measuring unit, not a skill you need to master before using AI.
AI does not count words the way you do
A token might be a whole short word, part of a longer word, punctuation, or another piece of text. Different models can split the same sentence differently, and different languages can produce very different token counts.
That is why “one token equals one word” is wrong. A decent rough estimate for ordinary English is that 100 tokens is somewhere around 60 to 80 words. Useful for intuition. Terrible for accounting.
You also do not need to look at a sentence and guess where every token boundary is. If an exact count actually matters, the provider has a tokenizer or token-counting tool that can do it properly.
The context window is basically the workbench
This is where tokens start to matter to normal people. Every model has a bounded amount of material it can actively work with at once. That working space is usually described as the context window, and it is measured in tokens.
Think of it less like permanent memory and more like the workbench for the current job. Your instructions, conversation history that is being included, pasted text, documents, tool results, and the answer itself can all compete for room in that working space depending on the product and model.
A bigger context window means the model can potentially work with more material at once. It does not automatically mean the model will notice every important detail, reason perfectly across all of it, or remember the information forever.
Context is the material available for the current work. Product memory, saved profile information, retrieval, and conversation compaction are separate mechanisms.
Why tokens keep showing up next to price tags
If you use a normal consumer AI app, you can mostly ignore token billing. Your subscription or usage limits may be presented in completely different terms.
If you build with an AI API, tokens become much more concrete because providers commonly price input and output based on token usage. Sending a giant document costs more than sending one paragraph. Asking for a huge answer costs more than asking for a short one. Reusing cached context can sometimes be priced differently too.
This is why developers suddenly become very interested in tokens while somebody using ChatGPT to rewrite an email can live a full and happy life without ever counting one.
Are tokens why a long chat sometimes starts acting weird?
Sometimes, partly — but “the AI ran out of tokens” is too simple an explanation for every weird long conversation.
As a thread grows, an AI product may have to decide what history to include, summarize, compact, retrieve, or leave out. Different apps handle that differently. Some products also have separate memory systems that can carry selected information across conversations.
The practical lesson is simpler: if you are doing important work over a long thread, do not assume every detail from page one is sitting equally fresh in front of the model forever. Keep the important constraints visible, ask for a checkpoint summary when the work gets complicated, and reattach the source material when accuracy matters.
Do normal people need to count tokens?
Usually? No.
You should start caring when you are stuffing very large documents into a model, building something with an API, debugging a context-limit error, or trying to understand why a massive prompt is expensive. Even then, the provider can usually count them for you.
For everyday AI use, the better habit is to care about useful context instead. Give the assistant the facts, constraints, examples, files, and boundaries that actually help it do the job. Remove giant repeated blocks that add nothing. Do not compress a good explanation into cryptic shorthand just because somebody on the internet told you shorter prompts are more advanced.
The one rule I would actually remember
Tokens are the ruler, not the goal.
You are trying to give the AI enough relevant material to do useful work while keeping the job understandable. Sometimes that is three sentences. Sometimes it is a 60-page PDF. Sometimes it is a photo plus one very specific question.
If you remember that tokens are just the chunks underneath all of this, you already know enough to understand most of the token talk you will hear. The rest is implementation detail you can learn only when you actually need it.