Tokenisation
Type any text and watch it split into the tokens a model actually sees — real BPE merges, token ids, and why models struggle to count the letters inside a word.
A language model never sees letters or words — it sees tokens: integer ids produced by a tokenizer that greedily merges frequent character pairs (Byte-Pair Encoding). Common words compress to a single token; rare words shatter into several.
This workspace runs a real BPE merge loop over an authored merge table on anything you type, live. Click any token to see the merge tree that built it, and load the "strawberry" preset to see exactly why a model that receives ids like [142, 97, 121] finds it hard to count the r's it never saw.
Token count is also what you pay for and what fills the context window — the workspace shows both, updating as you type.