Add structured conversation trace and export support for debugging and reproducibility

[tisha-varma]

Description

Currently, the Knowledge-Space Agent returns only the final answer for a user query, without exposing or persisting the intermediate steps involved in the retrieval-augmented generation (RAG) pipeline (e.g., retrieved documents, retrieval scores, prompts used).

This makes debugging, evaluation, and reproducibility difficult—especially for contributors and research-oriented use cases where understanding how an answer was generated is important.

This issue proposes adding optional structured conversation tracing and a mechanism to export conversation traces for analysis, while keeping the default user experience unchanged.

---

Proposed Solution

Backend

• Support an optional debug=true flag in chat requests.

• When enabled, collect and return structured metadata for each conversation turn, including:

  • Retrieved document identifiers, titles, and excerpts
  • Retrieval scores (when available)
  • Prompt text sent to the language model
  • Timestamps for major steps (retrieval, generation)

• Include this metadata under a new debug field in the API response, for example:

json
{
  "answer": "...",
  "debug": {
    "retrieved_documents": [...],
    "retrieval_scores": [...],
    "prompt": "...",
    "timestamps": { ... }
  }
} 

Frontend

• Add an “Export conversation” action in the chat UI.
• Allow downloading the full conversation as a JSON file containing:
  • User and agent messages
  • Associated debug metadata (when available)
• (Optional) Provide a collapsible UI section to view debug details inline for development and debugging purposes.

---

Documentation

Update developer documentation to describe:

• How to enable debug mode
• The structure and contents of the exported conversation trace
• Intended use cases such as debugging, evaluation, and reproducibility

---

Benefits

• Easier debugging of retrieval and response behavior
• Improved reproducibility for research and evaluation
• Better transparency into how answers are generated
• Useful for contributors without affecting production defaults

---

Scope

Medium — backend and frontend changes, additive only, with no changes to default behavior.

[SandeepChauhan00]

Hi @tisha-varma 👋

Really nice issue! Adding conversation tracing would definitely help with debugging and understanding how the agent generates its answers.

Just wanted to share a couple of things that might help:

• The chat_endpoint in backend/main.py already has a metadata field in the response, so you could extend that with a debug section when debug=true is passed.
• agents.py is where the RAG pipeline runs, so that's probably the best place to capture retrieved docs and scores.
• For the frontend export, App.tsx handles the conversation state, so adding a download button there should be straightforward.

Also, I just opened PR #56 which adds some logging and request tracking to the chat endpoint — some of that might be useful for your work too .

Let me know if you'd like to collaborate on any part of this. Happy to help !