Tokens: how a model reads (and why it bills you that way)

Every AI service bills "per token", and almost nobody knows what a token is. It's not a technical footnote: understanding it helps you estimate costs, write better instructions and see why a model sometimes fails at things that look silly, like counting letters.

Chunks, not words

Before reading your text, the model cuts it into chunks called tokens. Common words are usually a single token; long or rare ones split into two or three; punctuation goes on its own; numbers get cut into groups. In English, as a rule of thumb, one token is about four characters.

Type anything and watch how it gets cut. With "Read like the model", the frame walks through the text one chunk at a time, in the order the machine processes it:

Why it matters

Cost. APIs bill input tokens (what you send: instructions, history, documents) and output tokens (what it replies). An assistant that sends the whole catalog with every message spends ten times more than one that sends only what's relevant.

The limit. Every model has a maximum token window. When a long conversation fills it, the first thing to go is the beginning: the instructions you gave it.

Odd mistakes. Since the model sees chunks rather than letters, counting the "r"s in a word or spelling it backwards is harder than it looks. It isn't dumb: it has never seen the letters on their own.

Three habits that lower the bill

1. Write short, concrete instructions. Every extra word is paid on every message.
2. Don't send the full history: send a summary of what matters.
3. Search before you ask. Instead of pasting all your documents, send only the fragments that answer the question. That has a name, RAG, and it's the topic of the next post.