Kenhuang Academy
  • English

      • Embeddings

      An embedding turns text into a fixed-length float vector. Nearby meanings produce nearby vectors. RAG uses this for nearest-neighbor search—not for chat.

      Two primary paths in this course; neither requires a GPU:

      PathModelWhen to use it
      Cloud APIOpenAI text-embedding-3-small (1536-d by default)You have a key and want zero local weight files
      LocalOllama bge-m3, or BAAI models on the HubData stays on-box; Chinese / multilingual

      For Chinese retrieval, evaluate bge-m3 first (Ollama name bge-m3, Hub id BAAI/bge-m3). If RAM is tight, BAAI/bge-small-zh-v1.5 runs on CPU. See Ollama Embeddings for /api/embed, and Hugging Face for model cards and sentence-transformers.

      Index and query must use the same model. Changing models means re-embedding the store.

      Environment

      python -m venv .venv
source .venv/bin/activate          # macOS / Linux
pip install openai ollama numpy
      python -m venv .venv
.\.venv\Scripts\Activate.ps1
pip install openai ollama numpy

      OpenAI: set OPENAI_API_KEY. Ollama: install the app and ollama pull bge-m3 (CPU works; first run is slower).

      export OPENAI_API_KEY="sk-..."     # bash
ollama pull bge-m3
      $env:OPENAI_API_KEY = "sk-..."
ollama pull bge-m3

      OpenAI: text-embedding-3-small

      from openai import OpenAI

client = OpenAI()  # reads OPENAI_API_KEY


def embed_openai(texts: list[str]) -> list[list[float]]:
    resp = client.embeddings.create(
        model="text-embedding-3-small",
        input=texts,
    )
    return [item.embedding for item in resp.data]


vecs = embed_openai(["Refunds must be requested within 7 days of delivery."])
print(len(vecs[0]))  # 1536

      The API accepts dimensions to shrink vectors (e.g. 512) and save storage. Do not change dimensionality after you have indexed. Batch many strings in one input list; do not HTTP-loop one text at a time.

      Local: Ollama bge-m3

      No CUDA setup. Ollama can embed on CPU.

      import ollama


def embed_ollama(texts: list[str], model: str = "bge-m3") -> list[list[float]]:
    out = ollama.embed(model=model, input=texts)
    return out["embeddings"]


vecs = embed_ollama(["Refunds within 7 days.", "Support is online 09:00–18:00."])
print(len(vecs), len(vecs[0]))

      REST is POST http://localhost:11434/api/embed. The OpenAI-compatible path is POST /v1/embeddings. PowerShell:

      Invoke-RestMethod -Method Post -Uri "http://localhost:11434/api/embed" `
  -ContentType "application/json" `
  -Body '{"model":"bge-m3","input":"Why is the sky blue?"}'

      Do not embed with chat models (llama3, qwen, …): they are slow and not trained for retrieval.

      Local Python: BAAI / sentence-transformers (optional)

      If you want Hub weights without the Ollama daemon:

      from sentence_transformers import SentenceTransformer

# CPU is fine; bge-m3 is heavier—use bge-small-zh-v1.5 if RAM is tight
model = SentenceTransformer("BAAI/bge-small-zh-v1.5")


def embed_bge(texts: list[str]) -> list[list[float]]:
    return model.encode(texts, normalize_embeddings=True).tolist()

      pip install sentence-transformers pulls a CPU PyTorch wheel. The first model download needs disk and patience. A GPU helps but is not required.

      Hand-off to a vector store

      All three stores accept precomputed vectors:

      embeddings = embed_openai(chunks)          # or embed_ollama / embed_bge
# Chroma: collection.add(..., embeddings=embeddings)
# pgvector: INSERT ... embedding
# Qdrant: PointStruct(vector=embeddings[i])

      Chroma can also embed for you with its built-in function. For Chinese, pass your own OpenAI or bge-m3 vectors so you are not stuck with a tiny English default.

      Next steps

      评论