This page is that one thing, taken apart. Seven steps, each one running live in your browser — no API, no internet, no tricks. By the end you'll know what ChatGPT actually does when you press enter.
presenting? → next section · ← back · F fullscreen
Before a model sees your message, a tokenizer chops it into pieces from a fixed menu — common words stay whole, rare words get split, and anything unknown falls back to characters. Type anything:
Notice: "tokenization" isn't one token — it's three. Tamil text costs far more tokens than English, because the menu was built mostly from English text. This is why the same sentence costs different amounts in different languages.
Why it matters: everything downstream — cost, context limits, why models are bad at counting letters — starts here. A model never sees letters. It sees a sequence of numbers.
Each token ID is mapped to a list of numbers — a vector. Words used in similar ways end up near each other. Hover any word to see its neighbours, then run the famous arithmetic.
Honest label: real embeddings have thousands of dimensions, not two — this is a flattened sketch of the idea. But the arithmetic is real: in actual models, king − man + woman lands nearest to queen.
Why it matters: the model never stores definitions. "Meaning" is nothing but geometry — and that's why it can generalize to sentences it has never seen.
The 2017 breakthrough behind the T in GPT. Each token asks: "which other tokens matter for understanding me?" Hover any word — especially "it" — and watch where it looks. Then flip the last word.
What just happened: with "big", it attends to trophy; with "small", it attends to suitcase. No rule was written for this — it's learned. (Weights here are illustrative; a real model runs ~100 layers of these, with dozens of attention heads each.)
Why it matters: this is how the model handles context — pronouns, long-range references, "the report I sent you last Tuesday". Attention is the mechanism; everything else is plumbing.
Below is a real language model — a tiny one, trained on a 3 KB storybook corpus, computing live in your browser. The bars are its actual probabilities for the next word. Click a bar to choose it, and watch the loop.
Not faked: these probabilities are counted from the corpus as you watch. ChatGPT does the same thing with ~13 trillion training tokens and a vastly better model of context — but the loop is identical: predict, pick, append, repeat. (Did auto-complete get stuck repeating itself? That's greedy decoding — always taking the #1 token. The fix is the next section.)
Why it matters: once you see this loop, "hallucination" stops being mysterious. The model isn't lying — it's completing. Plausibility is the objective; truth is a side effect.
The model gives probabilities; sampling decides how to pick. Low temperature = always take the safest token. High = give underdogs a chance. Same model, same prompt — drag the slider and generate.
Try it: at 0.1 the model loops the same safe phrases. At 1.8 it rambles into nonsense. Real products live around 0.7–1.0 — and that's the whole "creativity" setting demystified.
Why it matters: the same model can feel like a careful lawyer or a drunk poet. When an AI product feels different, sometimes nothing changed but this number.
This is a neural network with 729 parameters learning to generate Indian names from scratch, using the same algorithm that trained GPT: guess, measure the error (loss), nudge every weight, repeat. Press train.
Watch the samples: step 0 is random letters. By a few thousand steps it invents names that don't exist but sound right — that is what "learning" means here: absorbing the statistics of the data. Nobody taught it rules.
Why it matters: GPT was trained exactly like this — just with 2.5 billion times more parameters and the internet as its name-list. The gibberish→fluency curve you just watched is the same curve, compressed from months into seconds.
Everything you just saw is the actual mechanism. The only thing separating this page from ChatGPT is scale — and scale, it turned out, was the magic ingredient.
Honest label: frontier-lab figures are reported/leaked, not official — treat them as order-of-magnitude. The point stands at any rounding: same loop, different zip code.
So when you use ChatGPT tonight, you'll know: your words became tokens, the tokens became vectors, attention decided what mattered, and a next-token loop — the one you clicked through yourself — wrote the answer one guess at a time. Now let's go build with it. → Session 1 starts here