#Ollama Integration

Point Dify at Ollama and keep both chat and embeddings off the public internet. Good for intranet demos and cheap iteration. Dify still owns orchestration, knowledge, and publish; Ollama is a local OpenAI-style HTTP API (default port 11434).

#Run Ollama on the host first

Install and start Ollama on the host (not inside the Dify containers). Pull a chat model and an embedding model, for example:

ollama pull qwen2.5:7b
ollama pull nomic-embed-text
ollama list

Names in Dify must match exactly. The Ollama tutorial covers install, Modelfiles, and VRAM. If Dify cannot reach a service bound only to 127.0.0.1, set OLLAMA_HOST=0.0.0.0 (or 0.0.0.0:11434) and restart Ollama. Do this only on a trusted network and firewall port 11434.

#Add the Ollama provider in Dify

1. Integrations → Model Provider, install Ollama
2. Setup: set the Base URL (next section—do not assume localhost)
3. Add models: chat models as LLM; nomic-embed-text as Text Embedding
4. Save and pass the connection check

Point default System Reasoning at the local LLM and default Embedding at the local vector model so new knowledge bases inherit them.

#Docker networking: localhost is wrong

Community Edition api / worker run in containers. Inside a container, localhost / 127.0.0.1 is that container, not your laptop. If Ollama is on the host, use:

http://host.docker.internal:11434

Smoke-test from the api container: curl http://host.docker.internal:11434/api/tags. If models list, go back to Studio and validate.

#Embeddings and knowledge

Local RAG needs the same embedding for index and query. A common pair:

• Chat: qwen2.5:7b / llama3.2 / …
• Vectors: nomic-embed-text or another embed model you actually pulled

Use High Quality indexing and select the Ollama embedding model. After you change embedders you must rebuild the index or retrieval silently degrades. Hybrid search still works; without a cloud reranker, tune weights and Top K first.

If VRAM is tight: stagger chat vs embed, use a smaller quant, or keep embeddings in the cloud and only run the LLM locally.

#Versus LangChain

LangChain uses langchain-ollama or an OpenAI-compatible base_url. Dify is the same URL in a form. Local context windows and tool-calling are usually weaker than flagship cloud models: check Function Calling on Agents, and drop to a Workflow if the local model cannot plan.

#Troubleshooting

Connection refused / timeout? Almost always a localhost mistake, Ollama bound to 127.0.0.1 only, or port 11434 blocked.

Validation 404? The model was never pulled, or the tag (:latest, etc.) does not match.

Knowledge processing fails? No embedding-type model, or worker also cannot reach host.docker.internal.

Still broken on Linux? Check extra_hosts, the host IP, and that Ollama listens on 0.0.0.0, not only the Docker bridge you guessed.

#Next steps

• Model Providers
• Knowledge Base
• Practical Examples
• Ollama tutorial