AI Studio — How LLMs Work

Watch how an LLM really works — type a prompt and see tokenisation, embeddings, attention, and next-token probabilities, all simulated in your browser.

All AI Topics (9)

  1. Tokenisation — Type any text and watch it split into the tokens a model actually sees — real BPE merges, token ids, and why models struggle to count the letters inside a word.
  2. Embeddings — Words as points in space: click any word to find its nearest neighbours by real cosine similarity, and run king − man + woman = queen as genuine vector arithmetic.
  3. Attention — Hover a word to see what it attends to — the real softmax(QKᵀ/√d) formula resolving “it” to “animal” or “street” depending on one word at the end of the sentence.
  4. Transformer — Step through the full architecture — embeddings, attention, MLP, residual connections, layernorm — watching one token’s vector transform at every stage on its way to logits.
  5. Chat Inference — Type a prompt and watch a chat reply generate token by token — logits, softmax, temperature, top-k, and the exact probability every token was sampled with.
  6. Training — Where the numbers come from: next-token prediction, cross-entropy loss, and a gradient-descent ball on a real loss surface — with a learning rate you can push until it diverges.
  7. Vector Databases — The same query answered two ways: brute-force scanning all 24 documents vs an IVF index that visits 9 — see exactly why approximate search is fast.
  8. RAG — Retrieval-Augmented Generation end to end — and the same question answered with retrieval switched off, so you can watch the model confidently make things up.
  9. Prompt Engineering — A system prompt isn’t magic — it’s tokens. Watch the same question’s next-token probabilities shift as you add a system prompt, few-shot examples, and chain-of-thought.