Kenhuang Academy
  • English

      • Installation

      Download the installer for your OS from ollama.com. Official platforms are macOS, Windows, and Linux. After setup, the API listens on http://localhost:11434 by default.

      Confirm the CLI

      Open a new terminal and check that ollama is on PATH:

      ollama --version

      Linux docs also use ollama -v. A version string means you can continue to Quick Start. If the command is missing: on macOS, keep the app in Applications and allow the /usr/local/bin/ollama link; on Windows, reopen the terminal or confirm %LOCALAPPDATA%\Programs\Ollama is on the user PATH.

      macOS

      • OS: macOS Sonoma (14) or newer
      • Hardware: Apple M-series (CPU + Metal); Intel Macs are CPU-only

      Mount ollama.dmg and drag the app into Applications. On first launch, if the CLI is not on PATH, macOS may ask to create a link in /usr/local/bin. Models and config live under ~/.ollama and can be tens to hundreds of GB. If the home volume is tight, place the app elsewhere and decline “Move to Applications,” or follow the official docs to relocate the model store.

      Logs: ~/.ollama/logs (app.log, server.log). Uninstall means removing the app, /usr/local/bin/ollama, and ~/.ollama plus related caches—see macOS.

      Windows

      • OS: Windows 10 22H2 or newer (Home / Pro)
      • GPU: NVIDIA driver 551.61+; AMD needs ROCm v7 / HIP7, or the default Vulkan path
      • OllamaSetup.exe does not require Administrator; it installs per user. The tray app stays running; ollama works in cmd and PowerShell

      Custom install directory:

      OllamaSetup.exe /DIR="D:\Apps\Ollama"

      Models default to %HOMEPATH%\.ollama. If that volume is small, set user environment variable OLLAMA_MODELS to a large disk, quit the tray app, and start Ollama again (or open a new terminal).

      Older Windows 10 console fonts may show progress as boxes—change the font. Some RDNA2 / RX 6000 cards lack ROCm v7 on current Windows drivers; Vulkan is the fallback. On mixed iGPU/dGPU boxes, GGML_VK_VISIBLE_DEVICES can pin the discrete GPU.

      Linux

      One-line install (recommended):

      curl -fsSL https://ollama.com/install.sh | sh

      The script installs the binary and usually sets up systemd. Manual tarballs, ARM64, and extra ROCm archives are in the Linux guide. Verify:

      ollama serve          # if it is not already a service
# other terminal
ollama -v

      Prefer a boot service: sudo systemctl enable --now ollama. Override env with sudo systemctl edit ollama. After NVIDIA drivers, nvidia-smi should list the GPU; AMD users should install current vendor drivers plus ROCm v7. Logs: journalctl -e -u ollama. Pin a release: curl -fsSL https://ollama.com/install.sh | OLLAMA_VERSION=0.5.7 sh (use a version from GitHub Releases).

      Docker (optional)

      Image ollama/ollama, port 11434, persist models:

      docker run -d -v ollama:/root/.ollama -p 11434:11434 --name ollama ollama/ollama

      NVIDIA needs the NVIDIA Container Toolkit and --gpus=all. AMD often uses ollama/ollama:rocm with /dev/kfd and /dev/dri. Run a model inside the container: docker exec -it ollama ollama run gemma4. When the app is in Docker and Ollama is on the host, see Integrations for host.docker.internal.

      GPU and memory (mental model)

      PlatformAccelerationNotes
      Linux / Windows + NVIDIACUDACompute capability 5.0+; driver 550+ (some older GPUs need 570+)
      Apple SiliconMetalUnified memory: weights count against system RAM
      Linux / Windows + AMDROCm or VulkanCheck the hardware list
      No discrete GPUCPUWorks; high latency. Start small or use q4

      Quantization vs VRAM (simplified): Library GGUF builds are often Q4 (~4 bits/weight) or Q8 (~8 bits). For the same parameter count, Q4 uses less VRAM with a small quality drop; Q8 is closer to the original and hungrier. Rough math: a 7B Q4 is about 4–5 GB, Q8 about 7–8 GB, plus KV cache for context. Longer context uses more VRAM; Ollama picks defaults from available VRAM (about 4k on smaller cards). ollama ps shows whether PROCESSOR is 100% GPU. Heavy CPU offload means the machine is overflowing.

      Limit NVIDIA devices with CUDA_VISIBLE_DEVICES. On CPU-only hosts, pull a small tag or use gemma4:cloud.

      Troubleshooting

      'ollama' is not recognized? New terminal; on Windows confirm the installer added the user PATH.

      Server up but curl fails? Open http://localhost:11434 in a browser; on Linux, systemctl status ollama.

      Disk full of weights? Set OLLAMA_MODELS and prune with ollama ls / ollama rm (Models).

      Next steps

      评论