Practical Examples
Three examples: a query engine, a FunctionAgent with tools, and multi-turn Context. LLM: gpt-4o-mini (historically the default was gpt-3.5-turbo; follow official docs). Set OPENAI_API_KEY and keep a data/ folder.
Agent-harness style: LangChain examples. Role crews: CrewAI. Concepts: RAG.
Example 1: Folder Q&A + persist
Goal: Answer questions over data/ without re-embedding on the second boot.
from pathlib import Path
from llama_index.core import (
Settings,
VectorStoreIndex,
SimpleDirectoryReader,
StorageContext,
load_index_from_storage,
)
from llama_index.llms.openai import OpenAI
from llama_index.embeddings.openai import OpenAIEmbedding
Settings.llm = OpenAI(model="gpt-4o-mini")
Settings.embed_model = OpenAIEmbedding(model="text-embedding-3-small")
PERSIST = "./storage"
if Path(PERSIST).exists():
index = load_index_from_storage(StorageContext.from_defaults(persist_dir=PERSIST))
else:
documents = SimpleDirectoryReader("data", recursive=True).load_data()
index = VectorStoreIndex.from_documents(documents)
index.storage_context.persist(persist_dir=PERSIST)
qe = index.as_query_engine(similarity_top_k=4)
response = qe.query("What is the overtime approval process? Use the notes.")
print(response)
for n in response.source_nodes:
print("-", n.metadata.get("file_name") or n.metadata.get("filename"), n.score)
Check: ask a proper noun that exists only in the files; missing OPENAI_API_KEY should fail; deleting storage/ should rebuild. Production can swap storage/ for Chroma.
Example 2: FunctionAgent with search + multiply
Goal: the model chooses document search vs arithmetic (full official starter shape).
import asyncio
from llama_index.core import VectorStoreIndex, SimpleDirectoryReader, Settings
from llama_index.core.agent.workflow import FunctionAgent
from llama_index.llms.openai import OpenAI
from llama_index.embeddings.openai import OpenAIEmbedding
Settings.llm = OpenAI(model="gpt-4o-mini")
Settings.embed_model = OpenAIEmbedding(model="text-embedding-3-small")
documents = SimpleDirectoryReader("data").load_data()
query_engine = VectorStoreIndex.from_documents(documents).as_query_engine()
def multiply(a: float, b: float) -> float:
"""Multiply two numbers."""
return a * b
async def search_documents(query: str) -> str:
"""Search local files for factual answers."""
return str(await query_engine.aquery(query))
agent = FunctionAgent(
tools=[multiply, search_documents],
llm=OpenAI(model="gpt-4o-mini"),
system_prompt="Use search_documents for corpus facts. Use multiply for arithmetic.",
)
async def main():
print(await agent.run("How do I request annual leave? Also, what is 12*8?"))
if __name__ == "__main__":
asyncio.run(main())
Check: math-only questions should skip search; policy-only questions should call search_documents. QueryEngineTool.from_defaults can replace the hand-written async function—Query Engines.
Example 3: Context remembers a name
Goal: turn two can answer “what is my name?” in the same session.
import asyncio
from llama_index.core.agent.workflow import FunctionAgent
from llama_index.core.workflow import Context
from llama_index.llms.openai import OpenAI
def echo_policy(topic: str) -> str:
"""Return a canned policy snippet for demos."""
return f"[demo] Policy about {topic}: submit via HR portal."
agent = FunctionAgent(
tools=[echo_policy],
llm=OpenAI(model="gpt-4o-mini"),
system_prompt="Remember the user's name across turns. Use tools for policy.",
)
async def main():
ctx = Context(agent)
await agent.run("My name is Alex. Please remember it.", ctx=ctx)
r = await agent.run("What is my name? What is the leave policy?", ctx=ctx)
print(r)
if __name__ == "__main__":
asyncio.run(main())
Check: omitting ctx on turn two should forget the name. Give each end user their own Context (or the persistence approach in official docs)—do not share one globally.
For a fixed retrieve → critique → rewrite loop, use @step in Workflows instead of stuffing control flow into one giant system prompt.
Anti-patterns
Next steps