Making AI
do things
Tool use: how a text-predictor gets hands — calculators, search, your files. Then the honest lesson the hype skips: when you need an agent, when you need a boring workflow, and the framework for choosing prompt vs RAG vs tools vs fine-tuning.
Still true after lunch?
Your RAG app is a brain in a jar
It can explain your notes beautifully. But ask it to:
Text in, text out — that's the whole jar. Today we wire the jar to the world. Carefully.
The model never executes anything.
It writes requests. Your code executes.
Tool use = the model outputs a structured function call ("call calculator with 2347 × 0.18") → your Python runs it → the result goes back into the context → the model continues writing. Still next-token prediction, end to end.
Which means: control stays with you. You decide which tools exist, validate every argument, and can refuse any call. Hold onto that sentence — Session 6 is built on it.
One tool call, step by step
Function declarations: the description IS the prompt
The model reads three things
Name, docstring, parameter types — that's its entire understanding of your tool. A vague docstring = wrong tool choices, missed calls, bad arguments.
"Use for ANY arithmetic — never compute numbers yourself" isn't documentation. It's an instruction to the model. Docstrings just became prompts.
The SDK converts Python functions to declarations automatically — types and docstrings do the work.
The industry-standard version of this menu is called MCP (Model Context Protocol) — a common format so any model can use any tool. Same idea you just learned, standardized.
The convenience hides two things worth seeing once. First, your Python function becomes a JSON schema — the same kind of schema you used to lock output shape in Session 3, now describing an input:
{"name": "get_marks", "description": "Look up a student's marks for one subject.", ← your docstring "parameters": {"type": "object", "properties": {"roll_no": {"type": "string", "description": "…"}, "subject": {"type": "string", "description": "…"}}, "required": ["roll_no", "subject"]}}That JSON — name, description, parameter names, parameter descriptions — is the model's entire knowledge of your tool. It cannot read the body. It cannot run it to see what happens. Which is why the slide says the docstring is the prompt: when a model picks the wrong tool, you have a writing bug, not a model bug.
Second, the conversation is no longer just user and model turns. Tool use adds two more roles, and they all stay in the context:
user
"What did roll 21C043 get in DBMS?"
model → functionCall
Not prose — a structured request naming the tool and its arguments. The turn ends here. The model has stopped and is waiting.
you → functionResponse
Your code runs the function and appends the result as a new turn. Nothing executed on the provider's side.
model → text
Now, seeing the result in its context, it writes the answer. Still pure next-token prediction over a longer transcript.
Two consequences that surprise people. Models can request several tools at once — if two lookups don't depend on each other, a good model returns both calls in one turn and expects both results back; running them in parallel is free latency you get by not assuming one call per turn. And tool use is a trained behaviour, not a feature you switch on: models are fine-tuned to emit these structured calls, which is exactly why a small model can be perfectly fluent in conversation and still poor at choosing tools. It's a separate skill with a separate score.
Because the whole exchange lives in the context window, every tool result you feed back is tokens you re-send on every subsequent turn. A tool that returns 5,000 tokens of raw JSON will quietly dominate the conversation and the bill. Return the smallest useful result — that is a design decision about the tool, not about the prompt.
MCP: USB-C for tools
You just wrote a tool declaration by hand — you built the socket. In 2024 the industry standardized the plug: the Model Context Protocol. One MCP server exposes your tools and data; any model that speaks MCP can use them.
Before MCP
Every app × every tool = hand-written glue. N apps, M tools → N×M integrations — a drawer full of proprietary chargers, one per phone.
After MCP
Write ONE server for your data or tool; Claude, Gemini, ChatGPT, your IDE all plug in. N+M — the way USB-C ended the cable drawer.
You already understand MCP — it's the function declaration you just wrote, standardized. That's exactly why this course builds raw first.
Which tool should it call?
The agent loop: while it wants tools, feed it
That while-loop is the whole "agent"
The model chose the order — nobody scripted read-then-calculate. Impressive. Also exactly where the danger lives → next slide.
Look hard at while. There is no exit condition except the model deciding it's finished. Ship that and you have written an unbounded loop whose termination is decided by a probabilistic system — with your API key attached to it. The real version of that line always carries a leash:
steps = 0 while response.wants_tool_call and steps < MAX_STEPS: assert response.call.name in ALLOWED_TOOLS # it can invent tool names result = execute(validate(response.call.args)) # and invent arguments response = model.continue_with(result); steps += 1And watch what happens to the context as that loop turns. Every call and every result is appended and re-sent, so the prompt grows monotonically: turn 10 might be re-reading nine tool outputs to decide the tenth action. Cost per step rises, latency per step rises, and — because models attend worst to the middle of long contexts — the early instructions start losing to recent tool noise. Long agent runs get more expensive and less obedient at the same time. Production systems fight this by summarising older steps and writing intermediate results to a file the agent can re-read on demand, rather than carrying everything in the window.
Now the score nobody keeps. Session 2 taught you to evaluate answers. An agent's answer being right is not enough — it can reach a correct answer through an absurd, expensive, or dangerous path, and you'd never know. So evaluate the trajectory, not just the destination:
Tool choice accuracy
Given a question, did it pick the right tool? Score this on its own with 20 fixed questions — it's the fastest signal that a docstring needs rewriting, and it needs no full run.
Step count
Track the distribution, not the average. A task that usually takes 3 steps and occasionally takes 15 is telling you it gets lost — and the tail is what burns your budget.
Unnecessary calls
Did it call a tool when it already had the answer, or call the same one twice with identical arguments? Both are common, both are pure cost, and both are invisible if you only grade the final text.
The cheapest observability you will ever add: log every tool call with its arguments, its result, and the elapsed time. When the agent misbehaves — and it will — that log is the difference between a five-minute fix and an afternoon of guessing. You'll build exactly this habit in Session 6.
Five ways tool use goes sideways
Wrong tool, confidently
"What year did TCE start?" → calls calculator. Why?
Vague docstrings + a model eager to use its toys. Fix: sharper descriptions + "answer directly when no tool is needed" in the system prompt.
Bad arguments
calculator("what is eighteen percent of 2347") — crash. Why?
It writes plausible args, not valid ones. Fix: validate/parse INSIDE every tool; return readable errors — the model actually corrects itself on a good error message.
The infinite loop
Search → weird result → search again → again… Why?
No termination judgment. Fix: hard cap (max 5 tool calls), then force an answer. Every agent framework has this number for a reason.
Imaginary tools
Calls send_email — which you never gave it. Why?
It's seen a thousand send_emails in training. Fix: execute ONLY menu functions; reject the rest. (Your code is the bouncer — another reason the model never executes.)
The fifth one is tomorrow's-session material, today
Tool results are text entering the context — a poisoned webpage or file can carry instructions: "ignore your rules and…". Session 6 weaponizes this. For now: treat tool results as untrusted input.
Most problems don't need an agent.
They need a boring workflow.
Workflow: YOU fix the steps in code (extract → validate → summarize → format). The model fills in the hard parts. Predictable, testable, debuggable.
Agent: the MODEL decides the steps at runtime. Flexible, impressive — and every extra decision is a new place to fail.
Morning idli at a Madurai mess is a workflow — same steps, every day, no decisions. A wedding feast is an agent — someone senior improvising under pressure.
The industry relearns this quarterly. Here's the arithmetic nobody puts on the poster →
Why long agent chains collapse
A 10-step agent at 95% per step is a coin flip you paid for.
The number to remember
Fixes, in order: fewer steps (workflow!), checkpoints with validation between steps, human approval on risky ones, and retries on cheap ones.
This one slide explains most "our agent demo failed in production" stories.
Multi-agent: more agents, more compounding
2026's favourite pitch: a crew of AI agents talking to each other. You now own the math to check it — every hand-off between agents multiplies the failure curve you just saw.
What actually ships
Specialist agents joined by a boring workflow: fixed hand-offs, validated at each step. The "crew" is usually a pipeline in a costume.
When true multi-agent wins
Genuinely parallel, independent subtasks: research fan-outs, red-team vs blue-team, many documents at once — with a judge step at the end.
Same rule as ten minutes ago: the fewer decisions the system improvises, the more often it works. More agents raise the ceiling AND the bill.
Most production “AI agents” are
a while-loop in a trench coat.
A model, a tool list, and while(model_wants_tools). That is the whole secret. The industry charges enterprise prices for that loop — you will write it yourself in the next fifty minutes.
Disagree? Good. Bring it to the break — strongest counter-argument gets named on the closing slide.
Workflow or agent? One question decides
Do you know the steps in advance?
Yes → workflow. "Every invoice: extract → validate → post." Script the steps, let the model do the smart parts inside each. This is most real business AI — by a wide margin.
Genuinely unpredictable path?
Then an agent — with a leash: max steps, tool allow-list, validated args, human sign-off on anything that writes, spends, or sends. (S6 makes this list official.)
Resume-honest phrasing: "built an AI workflow with tool use" beats "built an autonomous agent" in almost every real job. Interviewers have watched agent demos die at step 7 — they know this arithmetic.
The escalation ladder: cheapest fix first
—
—
Right approach? Defend your vote
In Lab Part D you'll do this in writing for three scenarios — that's a mini design doc, and it's capstone rehearsal.
API models vs models you own
Privacy & compliance
Some data legally can't leave your infrastructure — patient records, financial KYC. I live this at work in fintech: regulators ask exactly where every prompt goes. Local weights end the question.
Cost at scale
APIs bill per token forever; a small tuned local model can serve one narrow task at near-zero marginal cost.
Offline & edge
No internet in the field / on the factory floor / on a farmer's phone? The model must live on the device.
Open-weight models (Llama, Qwen, DeepSeek, Gemma) run on your hardware. The gap to frontier APIs is real but shrinking — and for a well-scoped task, often irrelevant.
Ollama: a model in your pocket
What you're watching
Gemma 4 E4B — a 4-billion-knob open-weight model (Apache 2.0, March 2026), quantized to fit in laptop RAM, running on this machine's GPU/CPU. Slower and less brilliant than the API — and it's yours: private, free per token, works on the Vaigai Express through a dead zone.
Try at home: 8 GB RAM runs 3–4B models fine. ollama.com — one installer, then the command above. (Last year's gemma3:4b still works too.)
You don't have to guess, and you don't have to download 8 GB to find out. A model file is just its parameters, so its size is parameters × bits-per-parameter ÷ 8:
4B params × 16 bits / 8 = 8 GB full precision — won't fit comfortably in 8 GB RAM 4B params × 8 bits / 8 = 4 GB Q8 — near-identical quality 4B params × 4 bits / 8 = 2 GB Q4 — the usual default, small measurable lossThat's quantization: store each knob in fewer bits. It works because the exact value of any one parameter barely matters — only the pattern across billions of them does — so rounding them is survivable in a way that deleting them is not. Q4 is the sweet spot almost everyone ships; below that, quality falls off quickly.
Two corrections to the arithmetic before you trust it. Add roughly 1–2 GB of headroom for the KV cache from Session 1 — it grows with your context length, which is why a long conversation can make a model that "fit" start swapping. And the number that decides your tokens-per-second isn't compute at all, it's memory bandwidth: generating each token requires reading every active parameter, so a 2 GB model on a laptop pushing ~50 GB/s tops out around 25 tokens/sec no matter how fast the CPU is. This is also precisely why Apple Silicon punches above its weight locally, and why quantization makes models faster as well as smaller — half the bits is half the reading.
Where quality actually drops
Not evenly. Quantized models hold up on chat and summarising, and degrade first on long multi-step reasoning, exact formats, and rarer languages — the hard tail, which is also the part a casual test never touches.
Test it, don't trust the size
You already own the tool: run your Session 2 eval set against the local model and the API model. One table settles "is local good enough for this job" in ten minutes, permanently.
Small ≠ dumb
A 4B model distilled from a frontier teacher, doing one narrow job with good prompts, routinely beats a much larger general model on that job. Fit the model to the task, not to the leaderboard.
Same arithmetic explains the data-centre side. A 70B model at Q4 needs ~35 GB — more than most single consumer GPUs — which is why serious self-hosting starts at multiple cards, and why "we'll just run our own" is a hardware budget conversation, not a weekend.
Local vs API: an engineering trade, not a religion
| Frontier API (Gemini/GPT/Claude) | Local open weights (Ollama) | |
|---|---|---|
| Capability | Best available | Good; excellent when task is narrow |
| Cost model | Per token, forever | Hardware once, then ~free |
| Privacy | Data leaves your machine | Never leaves |
| Upgrades | Provider upgrades free | You manage versions |
| Offline | No | Yes |
| Best first fit | Products, complex reasoning | Sensitive data, edge, high-volume narrow tasks |
Production pattern you'll actually meet: frontier API for the hard 10%, cheap local/small model for the routine 90%.
Six ideas you own now
Tool use
···
Model writes structured calls; YOUR code executes; result re-enters context. Control stays with you.
Declarations
···
Name + docstring + types = the model's entire understanding. Docstrings are prompts now.
Agent loop
···
While it wants tools, execute and feed back — with a hard step cap and an allow-list.
0.9510 ≈ 0.60
···
Reliability compounds against you. Fewer steps, checkpoints, human gates. Workflows beat agents for known tasks.
The ladder
···
Prompt → few-shot → RAG → tools → fine-tune. Escalate only when the cheaper rung measurably fails (evals!).
Local models
···
Open weights on your hardware: private, offline, ~free per token. Ollama makes it one command.
Lab 5: give it hands
Stretch
1. Wire search_notes from your Session 4 RAG app in as a tool — your capstone just became an assistant.
2. Write the agent loop manually — see every message.
3. Break it: make it call the wrong tool, then fix the docstring.
Capstone status check
After this lab you have: knowledge (RAG) + hands (tools) + a judge (evals). The final session hardens it and you demo. Almost there.
Your AI knows, sees, and now acts.
Final session: we attack everything you built — live prompt injection, poisoned documents — then harden it, talk real production costs, and you demo your capstone.