Kenhuang Academy
  • English

      • Knowledge Base

      A knowledge base is Dify’s RAG layer: chunk documents, embed them (or build an inverted index), retrieve passages at question time, and stuff them into the prompt. The idea matches LangChain RAG—Loader → Split → Embed → Retrieve → Generate—except Dify is a console flow, not Python classes.

      flowchart LR
  U[Upload docs] --> C[Chunk]
  C --> E[Embed / index]
  E --> V[Vector store]
  Q[User question] --> R[Retrieve]
  V --> R
  R --> P[Context]
  P --> M[LLM]
  M --> A[Answer + citations]

      Panel names drift between versions. The steps below are the flow, not a pixel-perfect click path.

      Create and import

      Open Knowledge, create a dataset, then import. Typical sources:

      • Local files (PDF, Markdown, TXT, … — check current docs for formats)
      • Website import and Notion sync (authorization required)
      • Advanced: a Knowledge Pipeline you orchestrate, or an external knowledge API in front of an existing RAG stack

      For the first corpus, upload a FAQ or policy you wrote. Plant proper nouns or fake clauses that do not exist on the public web, or you cannot tell retrieval from memorized pretraining.

      Chunk, index, retrieve (concepts)

      After upload you usually choose three things:

      1. Chunk mode

      Official modeMeaning
      GeneralAdjacent chunks by length / separators
      Parent-childRetrieve small, return large (docs often suggest this first)
      Q&AQuestion–answer pairs; good for canned FAQ

      Huge chunks add noise; tiny chunks lose context. Use the preview to see whether tables or headings were sliced badly.

      2. Index method

      MethodMeaning
      High QualityAn embedding model vectorizes chunks; vector / full-text / hybrid retrieval
      EconomicalInverted / keyword-heavy; fewer embedding tokens; weaker semantics

      Hybrid runs vector and full-text together, then merges by weights or a rerank model. Boost keywords for statute-like text; boost semantics for paraphrased questions.

      3. Retrieval filters

      Common knobs: Top K and a score threshold. A workflow Knowledge Retrieval node adds a second filter: the dataset recalls a pool, the node trims it. Multiple datasets search in parallel, then merge.

      When processing finishes, use the built-in retrieval test. Ask facts that exist only in the file. Empty or wrong hits: fix chunking and hybrid weights before you rewrite the app prompt.

      Attach to an app

      App typeTypical wiring
      ChatbotBind the dataset; each user turn is the query; hits are injected into the prompt
      Workflow / ChatflowAdd a Knowledge Retrieval node; pass a query variable; feed output to an LLM node
      AgentAttach datasets; the model decides whether to retrieve from each description (write those descriptions carefully)

      Tell the model: answer only from retrieved passages; say you do not know if evidence is missing; cite sources. Dify passes hits as context; UIs often show citations (exact chrome depends on Web App / node settings). Do not pretend the full document is in context—the model sees retrieved chunks only.

      Quality and ops

      PracticeWhy
      Proprietary test questionsRegression on facts only your docs contain
      Metadata filtersLimit search by team or product line
      Re-process after editsOld chunks go stale
      Citation auditsIn logs, check that cites actually support the claim

      Mapping to LangChain: dataset settings ≈ VectorStore + Retriever config; Knowledge Retrieval ≈ a retriever step on the chain; an Agent-attached dataset ≈ a retrieve tool the model may call.

      Troubleshooting

      Stuck on processing? Check embedding quota and that worker containers are healthy.

      Retrieval looks fine, answers drift? Raise the threshold, lower K, forbid uncited knowledge in the prompt.

      Cites are right, conclusion is wrong? The chunk was truncated—try parent-child or larger chunks.

      Next steps

      评论