Giving AI
your knowledge
The model has read the internet — but not your notes, your syllabus, your company's documents. Today: search that understands meaning, then RAG — the architecture inside nearly every "chat with your docs" product ever shipped. By lunch you'll have built "chat with my notes."
Did it survive the night?
Watch it not know — loudly
It cannot know — your syllabus isn't in its training data. Watch what it does instead of admitting that: a plausible OS syllabus, invented on the spot. Some models even apologise first — then produce a typical syllabus anyway. The hedge is not knowledge. Session 2's disease, aimed directly at your college life — today we cure it.
"Just paste all my notes!" — let's price that
Three taxes on pasting
1. The window — remember, everything must fit (Session 1). A textbook doesn't.
2. The meter — you pay per token, per question. Every "what's a deadlock?" re-sends all 400 pages.
3. The middle — even long-window models attend unevenly; whatever's buried mid-window is likeliest to be missed. More haystack, worse needle-finding.
Don't send everything.
Send the right 3 paragraphs.
Nobody reads the whole book in an open-book exam — toppers walk in with sticky flags already on the right pages. That's a good index. We need a search engine that finds paragraphs by meaning, staples them to the question, and lets the model answer from them.
Retrieval → Augmentation → Generation. RAG. Let's build the R first.
Keyword search misses meaning
Zero shared words — "marks/clear/subject" vs "50% aggregate/internals". Keyword search: nothing. But the meanings are neighbours… and you already know a machine that maps meaning to coordinates.
Embeddings, now for whole paragraphs
Same trick, bigger units
Session 1: words → coordinates (idli next to dosa). Today: entire chunks of text → coordinates. "Minimum 50% aggregate to pass" and "marks required to clear" land near each other in meaning-space — zero shared words needed.
Search = nearest neighbours
Embed all your chunks once (cheap). Embed the question. Find the chunks whose vectors sit closest — cosine similarity, one line of numpy. That's semantic search. That's the R in RAG.
The search playground
Chunking: how you cut the book
Too small → the retrieved fragment lacks context. Too big → the right sentence drowns, similarity dilutes. Paragraph-with-overlap is the boring default that wins. Chunking bugs cause more RAG failures than model choice does.
Vector databases, in plain words
What it is
A library shelved by meaning — hands back the k nearest vectors to any query, fast, even across millions.
Names you'll meet
FAISS (library), Chroma (dev-friendly), pgvector (Postgres extension), Pinecone/Weaviate (managed). Same idea, different packaging — like filter coffee from a tumbler, a flask, or a ₹300 café cup.
What YOU need today
A numpy array. Truly. Below ~100k chunks, one argsort is instant. Don't add a database until you have a database-sized problem.
Resume line, unlocked: "built semantic search over a vector store" — that sentence is the numpy array you'll write in the next 20 minutes.
RAG, end to end
The grounded prompt template
Three load-bearing lines
ONLY the context — blocks its internet-memories from overriding your documents.
Cite the chunk — users can verify; hallucinations become visible.
"I don't know" escape hatch — gives it a legal way out. Without one, it fills silence with fiction (you watched this at 9 a.m.).
Where RAG breaks in the wild
Right answer, wrong words
The user asks "attendance shortage rules" — notes say "condonation policy". Retrieval whiffs. Why?
Vocabulary gap even embeddings can miss. Fix: rephrase the query with the LLM first, retrieve more chunks (k=5), or index chunk summaries too.
The answer got cut in half
Rule starts in chunk 7, exception lives in chunk 8. Model sees only chunk 7. Result?
Confidently incomplete answer. Fix: overlapping chunks, or retrieve neighbours of every hit.
Stale index
Syllabus PDF updated Monday; embeddings made last month. What happens?
Confident answers from the OLD syllabus — with citations! RAG trusts its shelf. Fix: re-index on change; store doc dates; show them in answers.
Model ignores your context
Context says internal exam is 25 marks; the internet-average says 20. Model answers 20. Why?
Training memories leak past weak grounding. Fix: the ONLY line, lower temperature, and put context BEFORE the question. Test it — that's an eval (Session 2 never left).
RAG vs paste-it-all vs fine-tuning
| Approach | When it wins | When it loses |
|---|---|---|
| Paste into context | Few pages, one-off questions, quick scripts | Big corpora · repeated queries (you re-pay every time) |
| RAG | Large or changing knowledge · need citations · need freshness | Tiny static docs (overkill) · answers needing whole-corpus reasoning |
| Fine-tuning | Style, format, domain behaviour at scale | Teaching facts — expensive, freezes instantly, no citations. Common ₹-crore mistake. |
Interview one-liner: "Fine-tuning teaches behaviour; RAG provides knowledge." Full decision ladder after lunch.
You've been using RAG all along
Support chatbots
"Chat with our help docs." Retrieval over a manual + the grounded template.
NotebookLM-style tools
"Chat with your sources" — RAG with a polished UI.
AI search engines
Search results → context → cited answer. RAG at web scale.
Legal/medical assistants
Retrieval over case law / literature — citations are mandatory there.
"Chat with our wiki"
Every enterprise AI rollout starts here. Somewhere in Bengaluru, someone's entire salary is today's lab.
Yours, in 50 minutes
Chat with your own notes — same architecture, your data.
Your final-year project is
one grounded prompt away from being a startup.
Most funded "AI products" are exactly what you build today: retrieval plus a grounded prompt around someone else's model. That is not an insult to them — it is an invitation to you. The moat is the data and the evals, and you have Madurai data nobody in San Francisco has.
Disagree? Good. Bring it to the break — strongest counter-argument gets named on the closing slide.
Lab architecture: two pipelines
~60 lines of Python total. Every stage is something you've already touched this weekend.
Six ideas you own now
Chunk embedding
···
Whole paragraphs → coordinates. Meaning-neighbours, zero shared words needed.
Semantic search
···
Embed chunks once + question → cosine similarity → top-k. One numpy line.
Chunking
···
Paragraph + overlap. Too small = no context; too big = diluted. Causes more failures than model choice.
Augmentation
···
Staple top chunks into the grounded template: ONLY the context · cite · "I don't know" escape hatch.
RAG vs fine-tune
···
Fine-tuning teaches behaviour; RAG provides knowledge. Facts → RAG.
Fresh index
···
RAG trusts its shelf — stale embeddings give confident, cited, outdated answers. Re-index on change.
Lab 4: chat with YOUR notes
Stretch
Mini-eval: 5 questions with expected answers → score your RAG (harness from S2!) · k=1 vs k=3 vs k=5 · second document · show similarity scores in answers.
Capstone alert
This app is your capstone foundation. This afternoon adds tools (S5) and hardening + demo (S6). Build it on documents you actually care about.
Your AI now knows
what you know.
Next: it stops just answering and starts doing — calculators, search, your files. Tool use, agents, and the honest truth about both.