Reading a retrieval trace

A retrieval trace records one question: the reply the person received, every passage the retriever considered, how it scored, and whether the answer used it. It is where a thin or missing answer gets explained.

What you'll need

• The Knowledge Base app in your top bar.
• These permissions: Knowledge Base · Analytics - List, Knowledge Base · Analytics - Read.
• Questions already asked of this knowledge base. One trace is recorded per question.

If Knowledge Base isn't in your top bar, your role can't view it or your plan doesn't include it. Ask a workspace administrator, or see Permissions and module visibility.

Steps

1. Find the question

   Open the knowledge base and select Retrieval trace. The cards give the Avg. groundedness and the share Answered and Abstained. Rows give the Query, its Result, Groundedness, Retrieved, Used and when it was Asked. Result (All) filters every trace.

   

   The Retrieval trace section of Northwind Supplier Handbook: Avg. groundedness, Answered and Abstained cards, then seven questions, each with Answered or Abstained, Groundedness, Retrieved and Used counts and when it was asked.

   Enlarge

2. Read the summary and the reply

   Select the Query. The summary repeats the question with Answered or Abstained, how many passages were retrieved, used and access-filtered, a count for each reason passages were dropped, and Q&A shortcut with its score when a curated pair matched. Under it, Answer the user received shows the reply; Show full reply expands a long one.

   

   The trace of what documents does a new supplier need to send us?: Answered, 12 retrieved, 5 used, 0 access-filtered, 7 trimmed at rerank, the Answer the user received, then each candidate passage from supplier-onboarding-guide.pdf marked Used.

   Enlarge

3. See why it was withheld

   An abstained trace shows the Message the user received (answer withheld) and a callout headed Answer withheld with the reason, such as Below relevance threshold or No accessible content, and the score against what was required.

   

   The trace of what is the dress code for the summer party?: Abstained, the message the user received, and an Answer withheld callout reading Model found no answer in the passages.

   Enlarge

4. Scan the candidates

   Each passage lists its text, its Result, and its Rerank and RRF scores; select a score heading to sort by it. Result is Used, Not used, or why it was dropped: Below threshold, Trimmed at rerank, Deduplicated, Filtered · access or Injection blocked. The Result filter above the list narrows to one.

5. Expand one

   Select a passage for its full text, its Source with Open Source, its Candidate scores, whether a Q&A shortcut matched, and the question's Outcome.

   

   A candidate expanded: chunk 3 of supplier-onboarding-guide.pdf in full, its Source with Open Source, its Candidate scores, the Q&A shortcut section and the Outcome.

   Enlarge

What success looks like

• The counts account for every candidate: retrieved, used and access-filtered.
• Each candidate shows Used, Not used or a named reason.
• The reply the person saw is on the trace, and an abstained trace names its reason under Answer withheld.

Two drops to read differently

A red callout for access-filtered passages means the content exists but the person asking may not see it: an access question, not a weak match. See Who can see a knowledge base. A callout that the highest-scoring candidate was blocked by the injection guard means the best passage never reached the model, which often explains a poor or withheld answer.

Close and Run trace both take you back to the list; nothing is asked again.

If something goes wrong

Symptom	Likely cause	What to do
Retrieval trace is missing from the knowledge base's menu.	Your role has no Knowledge Base · Analytics permission.	Ask a workspace administrator.
A note says the reply was not captured for this trace.	The trace was recorded before replies were kept, or by a call that only retrieved.	Read the reply in chat, or ask again and open the newer trace.
No chunk text captured on every candidate.	An older trace, from before passage text was kept.	Ask the question again in chat, then open the newer trace.
Search by query… misses an older question.	Search only reads the 200 most recent traces.	Filter with Result (All) and page back instead.
Candidates show raw ids instead of file names.	Names come from the documents list, which your role cannot read or which holds more than 200 documents.	Match the id on Documents, or ask for Knowledge Base · Document - List.

Next

• Knowledge base analytics - Content gaps open a trace.
• Safety and guardrails - a guardrail decision opens its trace.
• Knowledge base settings - the strictness behind Below threshold and abstention.