Kenhuang Academy
  • English

      • Resources

      vLLM’s CUDA wheels, CLI, and quantization enums change quickly—prefer the official docs. This page collects those entry points, related courses on this site, and the repository.

      Official

      ResourceURL
      Docs homehttps://docs.vllm.ai/
      GitHubhttps://github.com/vllm-project/vllm
      Quickstarthttps://docs.vllm.ai/en/stable/getting_started/quickstart/
      GPU installhttps://docs.vllm.ai/en/stable/getting_started/installation/gpu/index.html
      OpenAI-compatible serverhttps://docs.vllm.ai/en/stable/serving/openai_compatible_server.html (newer trees may use serving/online_serving/)
      Dockerhttps://docs.vllm.ai/en/stable/deployment/docker/
      Parallelismhttps://docs.vllm.ai/en/stable/serving/parallelism_scaling/
      Quantization (e.g. FP8)features/quantization/ under https://docs.vllm.ai/
      Docker Hub imagehttps://hub.docker.com/r/vllm/vllm-openai

      If a path moves, search the title from the docs home.

      Facts this course repeats

      TopicConvention
      Installpip install vllm; happy path Linux + NVIDIA; Windows → WSL2
      CLIvllm serve <model>; legacy python -m vllm.entrypoints.openai.api_server --model ... (deprecated)
      HTTPhttp://localhost:8000/v1, chat /v1/chat/completions
      Demo modelsQwen/Qwen2.5-0.5B-Instruct, facebook/opt-125m (ungated)
      Multi-GPU--tensor-parallel-size
      Quant flag--quantization / quantization=; pre-quantized repos often auto-detect
      Imagevllm/vllm-openai
      CourseLinkWhen to read
      Ollama/en/ollama/Local DX, no server GPU, write the app first
      Hugging Face/en/huggingface/Repos, tokenizers, model cards, fine-tunes
      LangChain/en/langchain/Agents / RAG with base_url aimed at vLLM
      Docker/en/docker/Images, volumes, Compose, GPU runtime
      PyTorch/en/pytorch/Tensors and CUDA mental models

      Flow: Hub checkpoint → vLLM service → apps consume OpenAI protocol.

      Tutorial index

      1. vLLM Tutorial
      2. Introduction
      3. Installation
      4. Quick Start
      5. OpenAI-Compatible API
      6. Sampling
      7. Quantization
      8. Serving
      9. vLLM vs Ollama
      10. Docker
      11. Practical Examples

      Suggested exercises

      1. Hit Qwen 0.5B through both curl and the OpenAI SDK
      2. Compare temperature=0 vs 0.9 on one prompt
      3. Add --max-model-len to serve and watch VRAM
      4. Read an AWQ model card and try a load (read config only if VRAM is short)
      5. Start vllm/vllm-openai and call /v1/models from another terminal

      Contribute

      Part of Kenhuang Academy. When APIs or image names change, open an issue or PR and update both docs/zh/vllm/ and docs/en/vllm/.

      评论