We taught a machine to write Thirukkural. Here is everything that happened.

Every AI that writes does exactly one thing: guess what comes next.

Your phone's keyboard does it when it suggests the next word. Finish this: “Once upon a…” — you said time. That is the whole trick. Show the machine some text, it guesses what letter comes next, adds it, and guesses again. ChatGPT, Gemini, and the machine on this page all run this same loop. The difference is only how good the guess is.

Try it yourself first. Below is a real kural with one letter hidden. Guess it. Then see what the machine guessed — it has read exactly the same 1,330 couplets you are about to see.

Two things to notice. The machine never says “the answer is X.” It says “70% X, 20% Y, 10% Z” — it spreads its belief across options. And when it is unsure, it does not say “I don't know”; it just spreads the belief thinner. That spread is where everything interesting later comes from: creativity, mistakes, and the knob that makes it bold or cautious.

Everything the machine will ever know is in this one box.

Imagine a child who has read exactly one book in their life, and nothing else — no conversation, no school, no television. That is this machine. Its book is the 1,330 couplets below: 133 chapters of ten, in three parts — virtue (அறம்), wealth (பொருள்) and love (இன்பம்). Every couplet has the same shape: four words, then three words, then a full stop. The machine was shown only the poems — no meanings, no chapter names, no translations.

This gives the first honest rule about AI: the book is the ceiling. This machine cannot write about cricket, cannot use a letter that never appears in its book, cannot know who wrote the poems. The models behind ChatGPT feel unlimited only because their “book” is most of the public internet. The machinery is the same; the box is bigger.

Step one: give every letter a number, like a locker in a school corridor.

Line up every different letter in the book and number them 0, 1, 2 … 45. Now “அ” is just locker number 6, a space is locker 1, and a new line is locker 0. This numbered list is called the vocabulary, and each entry is a token. From here on the machine never sees a letter, only its locker number. Type anything below and watch it happen.

Why the big models use chunks, not letters

One number per letter is simple, but it makes every sentence long, and long is expensive. Big models instead learn a shorthand: find the two letters that appear together most often, glue them into one new token, and repeat a few thousand times. The result is a vocabulary of common chunks — syllables, then words. Run it on the kural text now and watch Tamil syllables assemble themselves.

Step two: every letter gets a place on a map. Meaning is where you stand.

Think of a spice rack. Where each jar sits tells you something: the ones you use together end up next to each other. The machine keeps a rack like that for letters — except the “place” is written as 64 numbers per letter instead of a shelf position. This is the embedding table. It starts completely random. Nobody tells the machine what a vowel is. But after learning, letters that behave alike sit close together, because that made guessing the next letter easier. Below is the real table and the real map.

Where in the line? A seat number for every letter

The map says what a letter is, not where it is. “அ” at the start of a poem and “அ” in the fortieth position look identical on the map. So the machine adds a second set of 64 numbers for the seat: seat 0, seat 1 … seat 127. Letter numbers plus seat numbers is what goes into the machine. Now it can tell “அ at the start of a line” from “அ after a space.”

Step three: every letter shines a spotlight back over the sentence and asks, “which of you matter to me?”

When you read “the bank was steep and muddy,” you know bank means a riverside, because you looked back at the words around it. The machine does that for every letter, all at once. Each letter shines a spotlight over everything before it; the letters that matter get lit brightly, the rest stay dim; and the letter blends together what it learned from the bright ones. This is called attention, and it is the invention that made today's AI possible.

Below is the machine's real spotlight on a real kural. Each row is one letter looking back; each dark cell is an earlier letter it lit up. The white triangle is the rule that a letter may not look forward — when the machine is writing, the future doesn't exist yet.

The arithmetic behind one spotlight (optional)

If you want to see the actual sums, pick a letter and step through what the first spotlight computes for it. These are the machine's own numbers, not a toy. If you don't, skip to the next chapter — you already have the idea.

The whole machine is one small routine, repeated three times.

One round of the routine has two parts. First, the spotlight: gather what matters from earlier letters. Second, a small “thinking” step that each letter does on its own: its 64 numbers go through a little calculator that mixes them and produces a new 64. “Mixes” means exactly this: each new number is a weighted sum of the old ones, anything negative is squashed toward zero, then a second weighted sum — 64 numbers widen to 256 and narrow back to 64, and every weight is a dial. In big models this is where facts about the world end up stored. Spotlight between letters, thinking within a letter. That is the whole division of labour. Do it three times — each round works on the richer picture the previous one left — and you are at the output.

Two housekeeping habits keep it stable. Each part adds its result to what the letter already had, rather than replacing it, so nothing learned earlier is thrown away. And before each part the 64 numbers are rescaled to a standard size so nothing blows up. Here is the whole design on one screen.

Where the 164,160 numbers live

Every arrow in the design is a table of numbers the machine is allowed to adjust while learning — think of them as dials. Count the dials and you have the machine's size. Ours has 164,160. Drag the sliders to see how a 164-thousand-dial poet becomes a 175-billion-dial chatbot: the same four ingredients, with two of them turned up.

Step four: score all 46 letters, then roll a weighted die.

The last letter's 64 numbers go through one final table and come out as 46 scores, one per possible next letter. The scores are turned into percentages that add up to 100. Then the machine does something people find surprising: it does not simply take the top letter. It rolls a die that is weighted by those percentages — a 70% letter wins seven times in ten. That is why the same question gets different answers each time.

Two knobs sit on that die. Temperature is how adventurous it is: turn it down and the favourite almost always wins (safe, repetitive); turn it up and unlikely letters get their turn (surprising, often wrong). Top-k simply throws away everything except the k best options before rolling. Type a few letters, move the knobs, and press write to watch the loop run.

Start with every dial at random. Guess, get corrected, nudge. 3,000 times.

This is how a child learns a song: sing it wrong, hear the right note, adjust a little, again. At the start every dial is random and the machine's guesses are noise. Then, over and over: take a random stretch of the book, cover the next letter, let the machine guess, uncover the truth, and measure how surprised it was. That surprise is the score to drive down — it is called the loss. A clever bit of maths (backpropagation) works out, for every one of the 164,160 dials, whether turning it slightly up or down would have made the surprise smaller. Turn each one a hair that way. Repeat.

If that sounds like magic, picture the slow version: wiggle one dial, re-run the guess, see whether the surprise fell, put the dial back, try the next one — 164,160 times for a single learning step. Backpropagation gets the same answer for every dial in one pass, by tracing the surprise backwards through the recipe with school calculus (the chain rule). That is its only trick, and it is why learning is affordable at all.

Nobody ever writes down a rule about Tamil. Spelling, grammar, the four-then-three rhythm, the full stop at the end — all of it is simply whichever dial settings made the surprise go down. Drag the slider and read what the machine was writing at each point of its education.

What one nudge looks like

Below are 64 of the dials, drawn as bars. Press one step and watch a learning step land: every bar moves a tiny, different amount, some up, some down. Nothing moves far. Thousands of small nudges in a consistent direction is the entire trick. (These bars are an illustration; the real dials are the ones you have been using in every box above.)

A machine that has only read continues. To make it respond, show it a few hundred examples of responding.

This is the gap between a model that completes text and an assistant that answers. The original GPT-3 could finish your sentence; ChatGPT answers your question. The difference is a short second education called fine-tuning: the same guess-and-nudge loop, but now on examples shaped exactly like the behaviour you want — a request, then the ideal response.

For our machine, the “request” is a chapter title. We rewrite the book so every couplet is preceded by a line naming its chapter, marked with a #, and continue the education for 1,500 more steps from where it left off. The machine picks up the new habit: after # அன்புடைமை (“possessing love”) comes a couplet in that mood. One new symbol and a format — that is all a “chat template” is.

அன்பிற்கும் உண்டோ அடைக்குந்தாழ் ஆர்வலர்
புன்கணீர் பூசல் தரும்.

அன்பிலார் எல்லாம் தமக்குரியர் அன்புடையார்
என்பும் உரியர் பிறர்க்கு.

# அன்புடைமை
அன்பிற்கும் உண்டோ அடைக்குந்தாழ் ஆர்வலர்
புன்கணீர் பூசல் தரும்.

# அன்புடைமை
அன்பிலார் எல்லாம் தமக்குரியர் அன்புடையார்
என்பும் உரியர் பிறர்க்கு.

Let the machine try twice, say which attempt was better, make that one more likely.

A teacher shows you the right answer. A judge only says “this one is better.” The last stage of building an assistant uses judges, and it is called RLHF — reinforcement learning from human feedback. Three moves. One: the machine writes two answers to a request; a person says which is better. Two: because people are slow and expensive, a second machine is trained to imitate their choices, so it can judge millions of answers nobody will read. Three: the writing machine is nudged so answers the judge prefers become more likely and answers it rejects become less likely.

Start with move one. You are the judge. Two couplets from the fine-tuned machine on the same topic: which is the better kural? Your clicks are exactly the kind of data that shaped ChatGPT — not answers, just preferences.

Move 2: a judge machine — and our honest stand-in

A real judge machine is another neural network trained on those clicks. This page cannot ship one that has learned from you, so it uses a simple rule that stands in for a human judging form: a good kural has exactly two lines, four words then three, ends with a full stop, and is not a copy of a real one. Pass or fail. Crude — it cannot judge meaning — and that crudeness will matter in a moment.

Move 3: the update, actually run

Starting from the fine-tuned machine: write two couplets for each of 600 random chapters; keep the pairs where the judge prefers one; then nudge the dials so the chosen couplet becomes more likely and the rejected one less likely — while staying close to a frozen copy of the machine we started from. That frozen copy is the leash. Without it the machine would race off toward whatever pleases the judge and forget everything else. The bars are the measured result: the share of fresh couplets that pass the judge, before the update and every 50 steps into it.

One letter, start to finish. Then again.

Everything above, in the order it actually happens. Press play and watch the machine fire once per letter as it writes a couplet. Nothing in the picture is decoration: the left column is the text so far; each tall column of lights is the newest letter's 64 numbers after a stage (blue positive, pink negative); the orange fan into each round is that round's real spotlight over the earlier letters; and the right column is the scoreboard for all 46 letters — the one the die picks lights up green and joins the text.

What size changes, and what it doesn't

Two things appear only at size: learning from the prompt itself (a big model picks up a pattern from examples you give it, with no dial changed — the basis of prompting, Session 2) and knowledge of the world (facts, stored in the “think” steps, which is what makes a model useful and what makes it confidently wrong). Our machine has neither, because 83 thousand letters contain neither.

The words you just earned

Nineteen words appear in every conversation about AI. You now own all of them, because you watched each one happen.

You can build this. Change the text file.

Everything on this page came from one text file and about 120 lines of code. To make a machine that writes something else, you replace the text file: Bharathiyar, Kabir's dohas, Ghalib, Shakespeare, Tamil film lyrics, your grandmother's recipes, your own WhatsApp group. If it is text, it works. Anything with a repeating shape — couplets, tweets, haiku, recipes — works well, because shape is the first thing a small machine learns.

What can go wrong, and what it means

Load the poet you built

The notebook's last step downloads three files — base.json, sft.json, judged.json. Choose any of them here and it runs in this page, on the same engine that runs the Thirukkural machine. Nothing is uploaded anywhere; the file stays in your browser.

For readers who program: the two files below are what trained every machine on this page.

Choices that matter at your size

Letters vs chunks. Under ~1 MB of text, stay with letters: chunks learned from tiny data are worse than none. Size of the machine. Match it to the book: 164k dials for 83k letters is already on the edge of memorising; a 1 MB book can carry twice the numbers per letter and 4–6 rounds. Stopping. Keep 10% of examples aside and stop when the machine's surprise on them turns upward. Temperature. The cheapest quality knob you have: 0.7–0.9 for poetry.

Tamil, Hindi, any Indian script. Letter-level treats a vowel sign (ி) as its own token, which is what you want: the machine learns the consonant-then-sign rule in its first hundred steps. Normalise the text (Unicode NFC) first so the same syllable is always the same bytes. English. Keep upper and lower case: it doubles the letters but teaches capital letters at line starts for free.