Making AI
do things
Tool use: how a text-predictor gets hands — calculators, search, your files. Then the honest lesson the hype skips: when you need an agent, when you need a boring workflow, and the framework for choosing prompt vs RAG vs tools vs fine-tuning.
Still true after lunch?
Your RAG app is a brain in a jar
It can explain your notes beautifully. But ask it to:
Text in, text out — that's the whole jar. Today we wire the jar to the world. Carefully.
The model never executes anything.
It writes requests. Your code executes.
Tool use = the model outputs a structured function call ("call calculator with 2347 × 0.18") → your Python runs it → the result goes back into the context → the model continues writing. Still next-token prediction, end to end.
Which means: control stays with you. You decide which tools exist, validate every argument, and can refuse any call. Hold onto that sentence — Session 6 is built on it.
One tool call, step by step
Function declarations: the description IS the prompt
The model reads three things
Name, docstring, parameter types — that's its entire understanding of your tool. A vague docstring = wrong tool choices, missed calls, bad arguments.
"Use for ANY arithmetic — never compute numbers yourself" isn't documentation. It's an instruction to the model. Docstrings just became prompts.
The SDK converts Python functions to declarations automatically — types and docstrings do the work.
The industry-standard version of this menu is called MCP (Model Context Protocol) — a common format so any model can use any tool. Same idea you just learned, standardized.
MCP: USB-C for tools
You just wrote a tool declaration by hand — you built the socket. In 2024 the industry standardized the plug: the Model Context Protocol. One MCP server exposes your tools and data; any model that speaks MCP can use them.
Before MCP
Every app × every tool = hand-written glue. N apps, M tools → N×M integrations — a drawer full of proprietary chargers, one per phone.
After MCP
Write ONE server for your data or tool; Claude, Gemini, ChatGPT, your IDE all plug in. N+M — the way USB-C ended the cable drawer.
You already understand MCP — it's the function declaration you just wrote, standardized. That's exactly why this course builds raw first.
Which tool should it call?
The agent loop: while it wants tools, feed it
That while-loop is the whole "agent"
The model chose the order — nobody scripted read-then-calculate. Impressive. Also exactly where the danger lives → next slide.
Five ways tool use goes sideways
Wrong tool, confidently
"What year did TCE start?" → calls calculator. Why?
Vague docstrings + a model eager to use its toys. Fix: sharper descriptions + "answer directly when no tool is needed" in the system prompt.
Bad arguments
calculator("what is eighteen percent of 2347") — crash. Why?
It writes plausible args, not valid ones. Fix: validate/parse INSIDE every tool; return readable errors — the model actually corrects itself on a good error message.
The infinite loop
Search → weird result → search again → again… Why?
No termination judgment. Fix: hard cap (max 5 tool calls), then force an answer. Every agent framework has this number for a reason.
Imaginary tools
Calls send_email — which you never gave it. Why?
It's seen a thousand send_emails in training. Fix: execute ONLY menu functions; reject the rest. (Your code is the bouncer — another reason the model never executes.)
The fifth one is tomorrow's-session material, today
Tool results are text entering the context — a poisoned webpage or file can carry instructions: "ignore your rules and…". Session 6 weaponizes this. For now: treat tool results as untrusted input.
Most problems don't need an agent.
They need a boring workflow.
Workflow: YOU fix the steps in code (extract → validate → summarize → format). The model fills in the hard parts. Predictable, testable, debuggable.
Agent: the MODEL decides the steps at runtime. Flexible, impressive — and every extra decision is a new place to fail.
Morning idli at a Madurai mess is a workflow — same steps, every day, no decisions. A wedding feast is an agent — someone senior improvising under pressure.
The industry relearns this quarterly. Here's the arithmetic nobody puts on the poster →
Why long agent chains collapse
A 10-step agent at 95% per step is a coin flip you paid for.
The number to remember
Fixes, in order: fewer steps (workflow!), checkpoints with validation between steps, human approval on risky ones, and retries on cheap ones.
This one slide explains most "our agent demo failed in production" stories.
Multi-agent: more agents, more compounding
2026's favourite pitch: a crew of AI agents talking to each other. You now own the math to check it — every hand-off between agents multiplies the failure curve you just saw.
What actually ships
Specialist agents joined by a boring workflow: fixed hand-offs, validated at each step. The "crew" is usually a pipeline in a costume.
When true multi-agent wins
Genuinely parallel, independent subtasks: research fan-outs, red-team vs blue-team, many documents at once — with a judge step at the end.
Same rule as ten minutes ago: the fewer decisions the system improvises, the more often it works. More agents raise the ceiling AND the bill.
Most production “AI agents” are
a while-loop in a trench coat.
A model, a tool list, and while(model_wants_tools). That is the whole secret. The industry charges enterprise prices for that loop — you will write it yourself in the next fifty minutes.
Disagree? Good. Bring it to the break — strongest counter-argument gets named on the closing slide.
Workflow or agent? One question decides
Do you know the steps in advance?
Yes → workflow. "Every invoice: extract → validate → post." Script the steps, let the model do the smart parts inside each. This is most real business AI — by a wide margin.
Genuinely unpredictable path?
Then an agent — with a leash: max steps, tool allow-list, validated args, human sign-off on anything that writes, spends, or sends. (S6 makes this list official.)
Resume-honest phrasing: "built an AI workflow with tool use" beats "built an autonomous agent" in almost every real job. Interviewers have watched agent demos die at step 7 — they know this arithmetic.
The escalation ladder: cheapest fix first
—
—
Right approach? Defend your vote
In Lab Part D you'll do this in writing for three scenarios — that's a mini design doc, and it's capstone rehearsal.
API models vs models you own
Privacy & compliance
Some data legally can't leave your infrastructure — patient records, financial KYC. I live this at work in fintech: regulators ask exactly where every prompt goes. Local weights end the question.
Cost at scale
APIs bill per token forever; a small tuned local model can serve one narrow task at near-zero marginal cost.
Offline & edge
No internet in the field / on the factory floor / on a farmer's phone? The model must live on the device.
Open-weight models (Llama, Qwen, DeepSeek, Gemma) run on your hardware. The gap to frontier APIs is real but shrinking — and for a well-scoped task, often irrelevant.
Ollama: a model in your pocket
What you're watching
Gemma 4 E4B — a 4-billion-knob open-weight model (Apache 2.0, March 2026), quantized to fit in laptop RAM, running on this machine's GPU/CPU. Slower and less brilliant than the API — and it's yours: private, free per token, works on the Vaigai Express through a dead zone.
Try at home: 8 GB RAM runs 3–4B models fine. ollama.com — one installer, then the command above. (Last year's gemma3:4b still works too.)
Local vs API: an engineering trade, not a religion
| Frontier API (Gemini/GPT/Claude) | Local open weights (Ollama) | |
|---|---|---|
| Capability | Best available | Good; excellent when task is narrow |
| Cost model | Per token, forever | Hardware once, then ~free |
| Privacy | Data leaves your machine | Never leaves |
| Upgrades | Provider upgrades free | You manage versions |
| Offline | No | Yes |
| Best first fit | Products, complex reasoning | Sensitive data, edge, high-volume narrow tasks |
Production pattern you'll actually meet: frontier API for the hard 10%, cheap local/small model for the routine 90%.
Six ideas you own now
Tool use
···
Model writes structured calls; YOUR code executes; result re-enters context. Control stays with you.
Declarations
···
Name + docstring + types = the model's entire understanding. Docstrings are prompts now.
Agent loop
···
While it wants tools, execute and feed back — with a hard step cap and an allow-list.
0.9510 ≈ 0.60
···
Reliability compounds against you. Fewer steps, checkpoints, human gates. Workflows beat agents for known tasks.
The ladder
···
Prompt → few-shot → RAG → tools → fine-tune. Escalate only when the cheaper rung measurably fails (evals!).
Local models
···
Open weights on your hardware: private, offline, ~free per token. Ollama makes it one command.
Lab 5: give it hands
Stretch
1. Wire search_notes from your Session 4 RAG app in as a tool — your capstone just became an assistant.
2. Write the agent loop manually — see every message.
3. Break it: make it call the wrong tool, then fix the docstring.
Capstone status check
After this lab you have: knowledge (RAG) + hands (tools) + a judge (evals). The final session hardens it and you demo. Almost there.
Your AI knows, sees, and now acts.
Final session: we attack everything you built — live prompt injection, poisoned documents — then harden it, talk real production costs, and you demo your capstone.