Level 1 · User
Tokens: why AI "forgets"
It reads in small pieces and can only keep so many in view.
AI doesn't read whole words but small pieces called tokens. In English, a word is roughly one token on average, a bit more for long words.
Each conversation has a limit of tokens the model can keep in view, called the context window. When a chat gets very long, the beginning can fall out. Tokens are also the unit you pay in.
Try it today
At the end of a long chat ask: "Summarize in 10 lines what we've decided." Paste that summary into a new chat.
Test yourself
Your chat is very long and AI no longer remembers the beginning. What do you do?
- Repeat everything in caps
- Open a new chat with a summary of what matters
- Switch computers
See the answer
Open a new chat with a summary of what matters. A short summary uses few tokens and leaves room to work.
If you lead a team
If your team pays per use, tokens are the bill. Long questions with whole documents cost more.