#Pipeline

RAG has two stages: index (offline) and query (online). Indexing turns documents into searchable vectors. Querying embeds the question, retrieves chunks, optionally reranks, then calls the LLM.

The chain this course owns:

flowchart LR
  D[documents] --> C[chunk]
  C --> E[embed]
  E --> V[vector store]
  V --> R[retrieve]
  R --> RR[optional rerank]
  RR --> L[LLM]

At query time the user question uses the same embed step, then retrieve. Do not embed with a chat model.

#What each stage does

Indexing can be batched or run overnight. Queries must be fast: vector search is milliseconds; the LLM dominates latency.

flowchart TB
  subgraph offline [Index]
    D[documents] --> C[chunk]
    C --> E[embed]
    E --> V[vector store]
  end
  subgraph online [Query]
    Q[question] --> EQ[embed]
    EQ --> R[retrieve]
    V --> R
    R --> RR[optional rerank]
    RR --> P[prompt + context]
    P --> L[LLM]
  end

#Responsibilities

1. documents — Markdown, extracted PDF text, database fields. Print plain text before you talk about retrieval.
2. chunk — Semantically whole, length-bounded pieces. See Chunking.
3. embed — Chunk → vector. See Embeddings.
4. vector store — Persist and search: Chroma, pgvector, Qdrant.
5. retrieve — Take k hits; add metadata filters or keyword hybrid if needed.
6. optional rerank — A cross-encoder or a rule that promotes true matches. See Retrieval.
7. LLM — Prompt: answer only from context; say you do not know if the context is silent.

Framework loaders / splitters / retrievers wrap these seven steps: LangChain RAG, LlamaIndex, Dify Knowledge Base.

#Smallest runnable sketch (in-memory, no database)

Chunk → embed → search → prompt. Swap the fake embed for OpenAI or Ollama; later chapters replace store with disk or Postgres.

from numpy import dot
from numpy.linalg import norm

def split(text: str, size: int = 80, overlap: int = 20) -> list[str]:
    chunks, i = [], 0
    while i < len(text):
        chunks.append(text[i : i + size])
        i += max(size - overlap, 1)
    return chunks

def fake_embed(s: str) -> list[float]:
    # Teaching only: production code calls the Embeddings chapter APIs
    vowels = sum(s.lower().count(c) for c in "aeiou")
    return [float(len(s)), float(vowels), float(s.count("refund"))]

def cos(a, b) -> float:
    return float(dot(a, b) / ((norm(a) * norm(b)) or 1.0))

raw = "Refunds must be requested within 7 days of delivery. Support is online 09:00–18:00."
chunks = split(raw)
store = [(c, fake_embed(c)) for c in chunks]
q = fake_embed("How long do I have to request a refund?")
top = sorted(store, key=lambda x: cos(q, x[1]), reverse=True)[:2]
context = "\n".join(c for c, _ in top)
prompt = (
    "Answer only from context. If unknown, say you do not know.\n\n"
    f"{context}\n\nQuestion: How long do I have to request a refund?"
)
print(prompt)

fake_embed must not ship. It only proves the data flow. Real vectors come from Embeddings.

#Which hop fails first

#Next steps

• Chunking
• Embeddings
• Chroma — first persistent pipeline