#Embeddings and a RAG teaser

Embeddings turn text into fixed-length float vectors for semantic search, clustering, and RAG. Dimension count is model-specific (often about 384–1024). Officially, /api/embed returns L2-normalized (unit-length) vectors, so cosine similarity is the usual metric.

Do not embed with a chat model: it is slow and not trained as a retriever. Pick an embedding model on ollama.com/library and ollama pull it.

#Pull a current embedding model

The capabilities page currently highlights embeddinggemma, qwen3-embedding, and all-minilm. You will still see these in the wild:

Names and tags change. Confirm on Library, then:

ollama pull embeddinggemma
# or
ollama pull nomic-embed-text
# evaluate for Chinese corpora as well
ollama pull bge-m3

CLI can emit a JSON array of floats:

ollama run embeddinggemma "Hello world"
echo "Hello world" | ollama run nomic-embed-text

Index and query must use the same embedding model or the spaces will not line up.

#REST: /api/embed

curl http://localhost:11434/api/embed -d '{
  "model": "embeddinggemma",
  "input": "The quick brown fox jumps over the lazy dog."
}'

Batch:

curl http://localhost:11434/api/embed -d '{
  "model": "embeddinggemma",
  "input": ["First sentence", "Second sentence", "Third sentence"]
}'

Optional fields: truncate, dimensions (if the model supports it), keep_alive.

Windows PowerShell:

Invoke-RestMethod -Method Post -Uri "http://localhost:11434/api/embed" `
  -ContentType "application/json" `
  -Body '{"model":"embeddinggemma","input":"Why is the sky blue?"}'

OpenAI-compatible: POST http://localhost:11434/v1/embeddings with model and input.

#Official libraries

import ollama

single = ollama.embed(
    model="embeddinggemma",
    input="The quick brown fox jumps over the lazy dog.",
)
print(len(single["embeddings"][0]))  # dimension

batch = ollama.embed(
    model="embeddinggemma",
    input=["First sentence", "Second sentence"],
)
print(len(batch["embeddings"]))

import ollama from "ollama";

const single = await ollama.embed({
  model: "embeddinggemma",
  input: "The quick brown fox jumps over the lazy dog.",
});
console.log(single.embeddings[0].length);

#Smallest retrieval intuition

1. Split documents (by heading or about 300–800 tokens).
2. embed each chunk; store vectors with the raw text (Chroma, FAISS, pgvector, …).
3. Embed the user question with the same model.
4. Take the top-k cosine hits, stuff them into a prompt, then chat / generate.

Sketch:

import ollama
from numpy import dot
from numpy.linalg import norm

def cos(a, b):
    return float(dot(a, b) / (norm(a) * norm(b)))

docs = ["Ollama listens on port 11434.", "Type /bye to leave the chat."]
vecs = ollama.embed(model="embeddinggemma", input=docs)["embeddings"]
q = ollama.embed(model="embeddinggemma", input="Which port?")["embeddings"][0]
best = max(range(len(docs)), key=lambda i: cos(q, vecs[i]))
print(docs[best])

For unit vectors, cosine equals a dot product. This is teaching code; production needs a real ANN index and metadata filters.

#Hook into LangChain RAG

The full pipeline (Loader → Splitter → VectorStore → Retriever → Agent) lives in this site’s LangChain tutorial:

• RAG
• Tool-using Q&A: Practical examples

You can swap OpenAI embeddings for Ollama embeddings and drive chat with ChatOllama or ChatOpenAI aimed at /v1, keeping data on-box. Dify knowledge bases that select an Ollama embedder follow the same “one model for index and query” rule—see Integrations and Dify.

#Watch-outs

• Changing the embedder means re-embedding the whole corpus.
• Judge Chinese quality on your documents, not only English leaderboards.
• Embedders occupy VRAM too; watch ollama ps if a 30B chat model shares the GPU.

#Next steps

• Integrations
• Practical examples
• LangChain RAG