Kenhuang Academy
  • English

      • Query Engines

      A QueryEngine is the high-level “ask the index, get an answer” API. The one-liner is index.as_query_engine(). Querying is three stages: retrieval → postprocessing → response synthesis. If you only need hits and will build the prompt yourself, use a Retriever from Retrieval.

      Pin synthesis with Settings.llm = OpenAI(model="gpt-4o-mini"). Generation historically defaulted to gpt-3.5-turbo; follow official docs.

      as_query_engine

      query_engine = index.as_query_engine()
response = query_engine.query("What did the author do in college?")
print(response)
print(response.source_nodes)  # which chunks were used

      Common knobs:

      query_engine = index.as_query_engine(
    similarity_top_k=5,
    response_mode="compact",
    verbose=True,
)

      Default top-k is small (docs often show 2). Raise similarity_top_k before adding a reranker. Async: await query_engine.aquery(...)—handy inside FunctionAgent tools.

      response_mode

      After retrieval, the synthesizer decides how chunks reach the LLM:

      ModeBehaviorUse when
      compactPack as many chunks as fit into one callDefault; fewer LLM calls
      default / refineOne LLM call per Node, refining the answerMore detail, more cost
      tree_summarizeRecursive tree reductionSummaries, multi-doc overviews
      no_textRetrieve only; inspect source_nodesDebug recall
      accumulateAnswer each chunk, then concatenateSame question against every span
      qe = index.as_query_engine(response_mode="tree_summarize", similarity_top_k=8)
print(qe.query("Summarize the data/ folder in three paragraphs."))

      Low-level: RetrieverQueryEngine

      When you need custom top-k and score cutoffs, assemble explicitly:

      from llama_index.core import get_response_synthesizer
from llama_index.core.retrievers import VectorIndexRetriever
from llama_index.core.query_engine import RetrieverQueryEngine
from llama_index.core.postprocessor import SimilarityPostprocessor

retriever = VectorIndexRetriever(index=index, similarity_top_k=10)
synth = get_response_synthesizer(response_mode="compact")
query_engine = RetrieverQueryEngine(
    retriever=retriever,
    response_synthesizer=synth,
    node_postprocessors=[SimilarityPostprocessor(similarity_cutoff=0.7)],
)
response = query_engine.query("What did the author do growing up?")

      A strict similarity_cutoff often means “no answer”—that is the point: empty is better than stuffing irrelevant chunks. Tune with no_text or by printing scores.

      Chat Engine: multi-turn over data

      QueryEngine is stateless by default. For follow-ups and coreference:

      chat_engine = index.as_chat_engine()
print(chat_engine.chat("What did the author do in college?"))
print(chat_engine.chat("Go on, more specifically."))

      chat_mode values (see current docs) include condensing history into a standalone retrieval query, or stuffing retrieved text into the system prompt. Streaming:

      streaming = chat_engine.stream_chat("One-sentence summary.")
for token in streaming.response_gen:
    print(token, end="", flush=True)

      When the model must decide whether to retrieve, prefer FunctionAgent + Context from Quick Start over stretching Chat Engine into an agent.

      Query engines as agent tools

      from llama_index.core.tools import QueryEngineTool, ToolMetadata
from llama_index.core.agent.workflow import FunctionAgent
from llama_index.llms.openai import OpenAI

tool = QueryEngineTool.from_defaults(
    query_engine=index.as_query_engine(similarity_top_k=4),
    name="handbook",
    description="Search the employee handbook. Use for policy questions.",
)
agent = FunctionAgent(
    tools=[tool],
    llm=OpenAI(model="gpt-4o-mini"),
    system_prompt="Prefer the handbook tool for factual policy answers.",
)

      You can also wrap aquery in async def search_documents(query: str) like the official starter. Multiple indexes (API ref vs tutorials) → multiple QueryEngineTools and let the model pick.

      Next steps

      评论