Practical Examples

Three examples, shallow to deep: an off-the-shelf sentiment pipeline, a retrieval-free chatbot, and a Trainer / LoRA outline. All models are ungated.

Example 1: Sentiment pipeline

Goal: Label short English reviews—enough for a moderation prototype.

from transformers import pipeline

sentiment = pipeline(
    "text-classification",
    model="distilbert/distilbert-base-uncased-finetuned-sst-2-english",
)

reviews = [
    "The pacing is perfect and the cast shines.",
    "I left the theater disappointed and tired.",
]
for text, pred in zip(reviews, sentiment(reviews)):
    print(f"{pred['label']:8} {pred['score']:.3f}  {text}")

Check: try obviously positive and negative sentences; confirm POSITIVE / NEGATIVE and the scores. For Chinese reviews, pick a Chinese checkpoint or fine-tune a Chinese DistilBERT—this SST-2 model is English.

Example 2: RAG-less chatbot (`generate`)

Goal: Multi-turn chat with no vector store. Use Qwen if you have a GPU; on CPU you can swap model_id to distilbert/distilgpt2 and drop the chat template (plain continuation).

from transformers import AutoModelForCausalLM, AutoTokenizer

model_id = "Qwen/Qwen2.5-0.5B-Instruct"
tok = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(
    model_id, device_map="auto", dtype="auto"
)
history = [{"role": "system", "content": "You are a brief, friendly tutor."}]

def ask(user: str) -> str:
    history.append({"role": "user", "content": user})
    prompt = tok.apply_chat_template(
        history, add_generation_prompt=True, return_tensors="pt"
    ).to(model.device)
    out = model.generate(
        prompt,
        max_new_tokens=96,
        do_sample=True,
        temperature=0.7,
        top_p=0.9,
    )
    text = tok.decode(out[0][prompt.shape[-1]:], skip_special_tokens=True)
    history.append({"role": "assistant", "content": text})
    return text

print(ask("What is a tokenizer? One sentence."))
print(ask("How does it relate to the model weights?"))

This is RAG-less: knowledge lives only in the weights and will hallucinate. For documents, see LangChain or your own retriever. For stable local chat, Ollama is less work.

Example 3: Fine-tune outline (Trainer or LoRA)

Path A — classification (shrink the data on CPU): follow Fine-tuning on distilbert/distilbert-base-uncased + rotten_tomatoes, call trainer.train(), then pipeline("text-classification", model=save_dir).

Path B — causal LM + LoRA (GPU recommended):

pip install peft
Prepare {"messages": [...]} or plain text; map to input_ids / labels
LoraConfig(task_type=TaskType.CAUSAL_LM, r=8, target_modules=["q_proj", "v_proj"])
get_peft_model, then Trainer with per_device_train_batch_size=1 and gradient accumulation if needed
push_to_hub only the adapter; the card must name Qwen/Qwen2.5-0.5B-Instruct as the base

VRAM: even 0.5B full FT can be tight; 7B full FT needs serious GPUs. On OOM, drop the batch or enable LoRA—do not blindly switch to Llama.

Anti-patterns

Anti-pattern	Do this instead
Ungranted Llama in examples	DistilBERT / DistilGPT2 / Qwen 0.5B
No `max_new_tokens`	Set it explicitly (see Generation)
Token pasted in a notebook	`hf auth login` or CI `HF_TOKEN`
7B training on CPU	Pipeline, Ollama, or a tiny Trainer slice
Shipping without reading the card	Check license and limitations first

Practical Examples

Example 1: Sentiment pipeline

Example 2: RAG-less chatbot (`generate`)

Example 3: Fine-tune outline (Trainer or LoRA)

Anti-patterns

Next steps

评论

#Practical Examples

#Example 1: Sentiment pipeline

#Example 2: RAG-less chatbot (generate)

#Example 3: Fine-tune outline (Trainer or LoRA)

#Anti-patterns

#Next steps

评论

Practical Examples

Example 1: Sentiment pipeline

Example 2: RAG-less chatbot (`generate`)

Example 3: Fine-tune outline (Trainer or LoRA)

Anti-patterns

Next steps