How machines
learned to talk
Zero scary math — one school formula, tops. A live demo for every big idea, and your first AI API call before you leave.
I sat in these seats — TCE CSE, class of 2009.
Karthikeyan NG. The seventeen years since: heading engineering at a fintech in Dubai, two startups founded, two books on machine learning, a TEDx stage, eighteen hackathon wins.
Not listed: the hundreds of things I built that failed — which taught me more than everything above.
Full story on the course site. That's all the bio you get — we have twelve hours, and you're here to build.
12 hours. 6 things you'll build.
S1 · First AI script
Your own API key, your own code calling a frontier model. Today.
S2 · Eval harness
A tester that measures whether AI answers are right — like an engineer, not a fan.
S3 · Vision app
Upload a photo, ask questions about it. AI that reads documents and images.
S4 · Chat with your notes
An AI that answers from your study material — the #1 real-world pattern (RAG).
S5 · Tool-using assistant
AI that calls a calculator, searches, reads files — and knows when not to be an agent.
S6 · Capstone demo
You attack each other's apps, patch the holes, and demo a real product.
Everything runs on free tiers, in the browser. No installs, no credit cards, no excuses.
Two kinds of AI: judges vs creators
The line that matters
Discriminative AI draws boundaries: spam or not, cat or dog, fraud or genuine. Input → label. It ran the world 2012–2022.
Generative AI produces new content: essays, images, code, answers. Input → new stuff. It's why you're in this room.
Same underlying math family — the difference is what the model is trained to output.
AI, ML, GenAI, LLM — who lives inside whom
—
—
App ≠ model
ChatGPT / Gemini app = the product you open. GPT-5.6 / Gemini 3.5 = the model inside it. Like WhatsApp vs the phone network — today in lab you skip the app and talk to the network directly.
ChatGPT is autocomplete
with a PhD.
Every large language model does exactly one thing: given some text, predict the most likely next token. Append it. Repeat. A thousand times per answer.
Essays, poems, code, exam answers — all generated one token at a time, left to right.
Sounds too simple to write poetry? Your brain plays this exact game every day — let's catch it in the act →
Play the model: guess the next word
Shout your guess before the reveal. Your brain has been doing next-token prediction since you learned Tamil.
Those odds aren't magic — you just count words.
The exact same three steps as Claude
1read the last word
2look up one row in the table
3sample the next word
1read the whole conversation
2one forward pass, billions of knobs
3sample the next token
Step 3 is identical — sampling is sampling. What changes is step 2, and training: counting hits a wall (vocabularyN rows), so an LLM learns a function instead of memorising a table.
Same idea, more machine: Markov published it in 1906, it ran your phone's autocomplete by 2010, transformers arrived in 2017.
Watch the network think
Honest scale: this toy has 21 neurons — frontier models have hundreds of billions of knobs and attention between every pair of tokens. But the choreography is exactly this: numbers in, numbers between, probabilities out, one token appended. Repeat.
Why the same question gives different answers
The model rolls dice
It doesn't always pick the #1 token — it samples from the distribution. Temperature reshapes those dice.
Low T → sharpens the peaks. Deterministic, safe, boring. Use for facts, code, extraction.
High T → flattens the field. Creative, surprising, riskier. Use for stories, brainstorms, names.
This is why your friend's ChatGPT answer ≠ yours. Nobody is lying. It's dice.
Wait — is it just searching a giant database?
No. And here's the proof: ask for something that has never been written by anyone, anywhere.
Nothing is stored, nothing is looked up
No sentences are saved as sentences — no articles, no answer bank. It computes every reply fresh, token by token, from learned patterns.
Fine print: text the internet repeats a million times can get burned into the knobs — but ask for anything new and it must compute.
Which raises the real question
If not lookup… where do those probabilities come from? Next slide: you'll build the answer with your own hands.
What's inside a model? Just knobs.
You remember y = wx + b
Those w and b are parameters — adjustable knobs. You're tuning them by eye to reduce the error. Training is exactly this, automated: guess → measure error → nudge knobs → repeat.
A “model” is nothing but a file full of learned knob values. You: 2 knobs, one line. Gemini: knobs in the hundreds of billions to a trillion — the labs won't say exactly — tuned on most of the internet.
Those probabilities in the guessing game? They come out of these knobs.
Training vs using: the cookbook rule
Training — writing the cookbook
Happens once, in the lab. Read trillions of tokens, nudge the knobs, for months, at hundreds of crores of cost. Ends with a frozen file.
Inference — cooking from it
Every time you chat. Your prompt flows through the frozen knobs, tokens come out in seconds. The book doesn't change while cooking.
Models don't read words. They read tokens.
Tokens are learned letter-chunks (~¾ of an English word each). The meter runs in tokens, not words — and the same sentence in Tamil can cost several times more than in English. Watch the meter run — then measure the real ratio on Gemini in the lab.
Embeddings: meaning becomes coordinates
Every token gets coordinates
The model turns each token into a long list of numbers — thousands of dimensions; this map shows 2. Words used in similar contexts land near each other.
idli ↔ dosa → neighbours.
idli ↔ GPU → different continents.
Famous party trick: king − man + woman ≈ queen. Meaning supports arithmetic.
Remember this map — in Session 4 it becomes a search engine over your own notes.
Attention: every word looks at every other word
One changed word flips what “it” means — and the model re-weights instantly. That mechanism is the T in GPT: the Transformer, from the 2017 paper “Attention Is All You Need.”
The whole trick, step by step
Every word of every ChatGPT answer went through this exact loop. A 500-word answer ≈ 650 loops.
If autocomplete can pass your exam,
your exam was never testing understanding.
You just watched the whole trick: no ideas, no intent, no understanding — only next-token statistics. If that is enough to top a question paper, the paper was measuring memory and pattern, not thought.
Disagree? Good. Bring it to the break — strongest counter-argument gets named on the closing slide.
The loop is old. The scale is new.
Quantity became quality. Keep scaling the same next-token loop and abilities nobody explicitly programmed start appearing — emergence; researchers still argue how sudden it really is. (And since 2024, labs also train models to think step-by-step before answering — scale isn't the whole story.)
How big did it get? ChatGPT crossed 1 billion monthly users in June 2026 — the fastest app in history to 1B, beating TikTok and Instagram. 900M were using it weekly back in February.
A freshly trained model is feral
A raw “base model” only continues text. It has read everything and learned nothing about manners. Try it:
A brilliant parrot of the whole internet — not an assistant. So how did it learn to answer instead of continue? →
Finishing school, in three steps
Pretraining
Read everything. Trillions of tokens, one game: predict the next one. Produces raw intelligence with zero manners.
Months · insanely expensive
Instruction tuning
Show it good behaviour. Many examples of “question → helpful answer” until it learns the assistant format.
The parrot learns to reply, not continue
RLHF
Teach it taste. Humans rank answers; the model is tuned toward what people prefer — helpful, harmless, honest.
Reinforcement Learning from Human Feedback
ChatGPT's 2022 breakthrough wasn't a smarter brain — it was better finishing school on top of the same loop.
Reasoning models: think longer, not just train bigger
Since late 2024 there's a third way to buy intelligence. Dial 1 was scale — a bigger brain. Dial 2 was finishing school — better manners. Dial 3: let the model spend tokens thinking before it answers. A scratchpad you usually don't see, then the reply.
What thinking buys
Hard math, multi-step logic, tricky debugging. On the scratchpad it tries a path, spots its own mistake, and backs up — it catches its errors before you ever see them.
Still next-token prediction — just aimed at itself first
What thinking costs
10–50× more tokens — real money, real seconds. Rule of thumb: easy question → fast model. Genuinely hard reasoning → thinking model.
Never pay thinking prices for “what's the capital of Tamil Nadu?”
Same autocomplete, one new habit: it learned to use scratch paper. (Frontier flash models expose this “thinking” mode — more tokens before the answer, so more accuracy and more cost.)
Why ChatGPT ≠ Gemini ≠ Claude
Three kitchens, same recipe book, different biryani: different training data, different finishing school, different house rules (system prompts) and tools bolted on. The map as of mid-2026:
| Lab | Flagship family | Weights | Worth knowing |
|---|---|---|---|
| OpenAI | GPT-5.6 (Luna · Terra · Sol) | Closed | Started the ChatGPT era |
| Google DeepMind | Gemini 3.5 Flash · Pro | Closed | 1M-token windows · our free lab |
| Anthropic | Claude Fable 5 · Sonnet 5 · Opus 4.8 | Closed | Safety lab · coding + long docs |
| Meta | Llama | Open | Weights you can download |
| Open-weights wave | Gemma 4 · Qwen · DeepSeek | Open | Frontier-chasing open models |
“Open weights” = you download and run the model yourself — we do that with Ollama in Session 5. In today's lab you'll taste the kitchens: identical prompt, three models. Names change monthly; the mechanics don't.
Famous failures — you explain them
Click a card only after the class has guessed the mechanism.
“How many r's in strawberry?” → “2”
Models kept getting this wrong for years. Using today's ideas — why?
Tokens. The model sees str·aw·berry as chunks, not letters. It literally cannot see the r's. Today's models finally pass — because labs drilled them on exactly this. The mechanism (tokens, not letters) hasn't changed.
847 × 923 = confidently wrong
A plain model, no calculator tool: it aces the explanation but flubs the arithmetic. Why?
Prediction ≠ calculation. It predicts digits that look plausible. Fix: hand it a calculator — tool use, Session 5.
Turn search off — clueless about yesterday's match
It knows last year in detail but not last night. Why?
Knowledge cutoff. Training ended months ago. Fix: give it search or your documents — RAG, Session 4.
Invents a convincing fake citation
Real journal, real-sounding authors, paper doesn't exist. Why?
Hallucination. It optimizes for “sounds likely,” not “is true.” Fix: evaluation + grounding — Session 2 is our lie detector.
None of these are bugs. They're direct consequences of next-token prediction — each with an engineering fix you'll build in this course.
The context window: its entire working memory
If it isn't in the window, it doesn't exist — the model's entire universe is one box measured in tokens. Try filling it:
The twist: it remembers nothing between turns. The app re-sends everything, every turn.
That’s the whole illusion of “chat memory” — the app quietly stuffs the entire conversation back into this window each time you hit send, which is why long chats get slow and forgetful.
And when your knowledge doesn’t fit? That’s Session 4: RAG — the pattern behind most real AI products.
Eight words you own now
Active recall: say the definition out loud before clicking.
Token
···
The chunk models actually read. You pay per token; Tamil costs more than English.
Embedding
···
Token → coordinates in meaning-space. Similar meaning = nearby points.
Attention
···
Every word weighs every other word to resolve meaning — the Transformer's engine.
Prediction
···
One task only: score every possible next token. The whole loop, repeated.
Sampling
···
Pick from the scores with temperature dice — why answers differ run to run.
Tuning
···
Pretraining → instruction tuning → RLHF: parrot goes to finishing school.
Parameter
···
One learned knob. A model = a file of billions of them. No sentences stored, no lookup.
Inference
···
Using the frozen model. It doesn't learn from your chat — and it has no memory between turns.
If you can explain these eight to your roommate tonight, Session 1 worked.
What actually happens when you call an API
Key = your identity
It says who's asking and whose quota to use. Treat it like a password.
429 = slow down
Typical free-tier limit ≈ 10 requests/min — check the live limits page. Our notebook auto-retries politely.
Latency is honest
1–3 s = the GPUs literally running one loop per token of your answer.
Your lab kit, and the rhythm
Lab handout
Parts A → E, top to bottom. Exact steps, checkpoints, and stretch goals — you never have to wonder “what now?”
Colab notebook
Cells pre-written, you run and edit. File → Save a copy in Drive first, every time.
Cheatsheet
One page per session — key terms + the code that matters. Keep it open in a tab; it's your revision material too.
Everything lives at
tinyurl.com/tce-genai
Bookmark it now — decks, handouts, notebooks, cheatsheets for all 6 sessions.
10 min
show me
don't skip!
The rhythm — every lab, all weekend
1. Pairs — both partners run every cell.
2. Hit a ✓ checkpoint → wave me over, show it.
3. Stuck > 5 minutes → ask the pair beside you first, then me.
4. Finished early → stretch goals. There is always more.
Checkpoints are how I know nobody's silently left behind — they're for you, not for marks. Using an AI assistant to write your lab code? Allowed — encouraged. One rule: your eval set judges its code too.
Lab 1: your first AI API call
gemini-flash-latest is our free workhorse — the alias gives you the newest Flash automatically — today that's Gemini 3.5 Flash. The free model in your lab today beats last year's flagship.
Rules of the lab
Work in pairs, both run the code. Stuck > 5 min? Ask the pair beside you first, then me. Finished early? Stretch goals on the handout.
Two things before the next session
Your 10 questions — written in Part E
You wrote these in Part E: 10 questions on a subject you know cold — DSA, cricket stats, Ilaiyaraaja discography — with the correct answers saved in a text file. That file is live ammunition for the lie-detector lab, next session. Missing it? Two minutes. Write it now.
Coming up: you catch it lying
In the next session's lab you'll interrogate the model on your expert topic until it confidently states something false — then measure how often, with numbers. Best catch gets bragging rights.
Next session — after a short stretch
Talking to AI, and Catching Its Lies — prompt engineering that actually works, and how engineers measure AI quality with numbers instead of vibes. Keep your laptops open and your API keys warm.
You've learned the trick.
Next: you learn to drive it.
Stretch, refill water, and find me if anything from this session is still fuzzy:
Deck + lab + cheatsheet are in the course repo. Keys are secrets — never commit them.