Practical examples
Three standalone examples: a terminal Q&A loop, an OpenAI-compatible client, and a custom coding assistant. Finish Installation and ollama pull gemma4 (or your stand-in) first.
Example 1: Local Q&A CLI
Goal: Multi-turn questions in the terminal. History stays in-process. Empty line or /bye exits.
#!/usr/bin/env python3
from ollama import chat
MODEL = "gemma4"
messages = [
{
"role": "system",
"content": "You are a local knowledge assistant. If you do not know, say so. Do not invent URLs.",
},
]
print(f"Local Q&A (model {MODEL}). Empty line or /bye to quit.")
while True:
user = input("you: ").strip()
if user in {"", "/bye"}:
break
messages.append({"role": "user", "content": user})
reply = chat(model=MODEL, messages=messages)
text = reply.message.content
messages.append({"role": "assistant", "content": text})
print(f"assistant: {text}\n")
pip install ollama
python qa_cli.py
Check: Mention a proper noun in turn one; turn two should still know it. Change models by editing MODEL. Persist history by writing messages to JSON. For local documents, start with Embeddings, then LangChain RAG.
Example 2: Mini OpenAI-compatible Python client
Goal: Use only the openai package against /v1, so you can point the same code at a vendor later.
from openai import OpenAI
client = OpenAI(base_url="http://localhost:11434/v1/", api_key="ollama")
def ask(prompt: str, model: str = "gemma4") -> str:
stream = client.chat.completions.create(
model=model,
messages=[
{"role": "system", "content": "Answer in short, clear bullets."},
{"role": "user", "content": prompt},
],
stream=True,
)
chunks: list[str] = []
for event in stream:
delta = event.choices[0].delta.content or ""
print(delta, end="", flush=True)
chunks.append(delta)
print()
return "".join(chunks)
if __name__ == "__main__":
ask("List Ollama's local API base and the OpenAI-compatible base.")
Windows without Python—smoke the same route:
Invoke-RestMethod -Method Post `
-Uri "http://localhost:11434/v1/chat/completions" `
-ContentType "application/json" `
-Body '{"model":"gemma4","messages":[{"role":"user","content":"Say this is a test"}]}'
Check: ollama ps should show gemma4 while it runs. A wrong port should fail immediately—proof you are not hitting public OpenAI.
Example 3: Custom Modelfile coding assistant
Goal: Freeze a code-review persona and a low temperature for Cursor and the CLI.
Modelfile:
FROM gemma4
PARAMETER temperature 0.2
PARAMETER num_ctx 8192
PARAMETER num_predict 1024
SYSTEM """You are a careful coding assistant.
- Name defects first, then the smallest patch.
- Do not invent APIs that are not in the prompt.
- Assume Python 3.11+ and PowerShell 7 unless told otherwise.
- Keep identifiers in English; prose may match the user.
"""
ollama create kenhuang-coder -f Modelfile
ollama run kenhuang-coder
Native API smoke test:
curl.exe http://localhost:11434/api/chat -d "{\"model\":\"kenhuang-coder\",\"stream\":false,\"messages\":[{\"role\":\"user\",\"content\":\"Review: def add(a,b): return a+b\"}]}"
In Cursor, set the model to kenhuang-coder and the base URL to http://localhost:11434/v1 (Integrations).
To switch the base weights (for example Library qwen2.5), change FROM and create a new name so you do not overwrite the old one.
Anti-patterns
Next steps