Introduction

What is RAG?

RAG (Retrieval-Augmented Generation) finds relevant snippets from your documents before calling the LLM, then sends those snippets plus the question to the model. Generation is still a language model. Company policy, product FAQs, and yesterday’s API change come from retrieval—not from pretraining.

Without retrieval, the model only has training data and whatever you stuffed into the prompt. A larger context window does not make it cheap to paste an entire knowledge base on every turn, and private facts never appear in public weights.

question ──► retrieve (vectors / keywords) ──► chunks ──► prompt + LLM ──► answer

Why RAG?

Pain	How RAG helps
Hallucinations	Instruct the model to speak only from retrieved chunks, with citations
Private knowledge	Policies, tickets, internal wikis—no fine-tune required
Freshness	Edit Markdown / a table → re-chunk and re-embed; no retraining
Audit trail	You can show which file and chunk a sentence came from

When not to force RAG:

The whole corpus is two or three pages—paste it into the prompt
You need writing style or task skill, not fact lookup → consider fine-tuning (Hugging Face Fine-tune)
Millisecond latency and a frozen corpus → a cache or rules engine may win

RAG vs fine-tuning vs long context

Approach	Strength	Cost
Long context	One-shot tasks, short docs	Tokens; knowledge is not reusable across sessions
RAG	Large, frequently updated facts	You maintain chunks, embeddings, indexes, eval
Fine-tuning	Format, tone, task skill	Data and training cost; facts need another train run

You can combine them: RAG for facts, fine-tuning for output shape, long context for this turn’s retrieved chunks. This course only owns the RAG path.

This course: principles, not one framework

The same pipeline shows up as:

Shape	Course on this site
Plain Python (this course)	Chunk functions + embedding APIs + Chroma / pgvector / Qdrant
Framework wrappers	LangChain RAG, LlamaIndex
Console	Dify Knowledge Base
Local embeddings	Ollama Embeddings

Once the principles are solid, switching frameworks is mostly renaming Loader / Retriever classes. Choosing a store (local files, Postgres, a dedicated service) stays the same problem.

Vocabulary

Term	Meaning
Chunk	Smallest retrieved unit, typically a few hundred to ~1k tokens
Embedding	A fixed-length float vector; nearby meanings → nearby vectors
Vector store	Vectors + source text + metadata, plus nearest-neighbor search
Retrieve	Take top-k chunks by query vector (or keywords)
Rerank	Score the coarse list again so true matches rise
Generation	The LLM answers after reading the chunks

Index (offline) and query (online) must use the same embedding model. Changing models means re-embedding everything. Do not trust English leaderboards for Chinese corpora—measure on your FAQ.

Smallest intuition

No vector database yet—just “nearest neighbor.” Production code should use the ANN indexes in later chapters.

from numpy import dot
from numpy.linalg import norm

def cos(a, b) -> float:
    return float(dot(a, b) / (norm(a) * norm(b)))

docs = ["Refunds must be requested within 7 days of delivery.", "Support is online 09:00–18:00."]
# Fake vectors: a real system uses an embedding API here
vecs = [[0.9, 0.1], [0.1, 0.9]]
q = [0.85, 0.2]
best = max(range(len(docs)), key=lambda i: cos(q, vecs[i]))
print(docs[best])

On unit-length vectors, cosine and dot product match. OpenAI and Ollama embeddings are often already normalized—check the vendor docs.

Next steps

Pipeline — connect the table into one diagram
Chunking — bad chunks poison everything downstream
Embeddings — cloud or local; no GPU required

#Introduction

#What is RAG?

#Why RAG?

#RAG vs fine-tuning vs long context

#This course: principles, not one framework

#Vocabulary

#Smallest intuition

#Next steps

评论