Attention
Hover a word to see what it attends to — the real softmax(QKᵀ/√d) formula resolving “it” to “animal” or “street” depending on one word at the end of the sentence.
Attention is how a transformer lets every token look at every other token and decide which ones matter for its meaning. Each token asks a question (its Query), offers an answer (its Key), and the match — softmax(QKᵀ/√d) — decides how much information flows.
The classic demonstration: "The animal didn't cross the street because it was too tired." What does "it" refer to? Change "tired" to "wide" and the answer flips from animal to street. Hover "it" in both sentences here and watch the attention arcs flip — the matrix is genuinely computed with the real formula over authored Q/K vectors.
Two heads are included: one behaving positionally, one resolving references — showing why transformers use many heads in parallel.