-
Notifications
You must be signed in to change notification settings - Fork 2
Description
Problem
When the EEGLAB assistant gives a suboptimal answer (e.g., missing the standalone erpimage function), developers have no way to trace what happened during the request:
- Which tools did the model call?
- What arguments were passed?
- What did each tool return?
- Did a tool return a fallback/error message?
- Why did the model choose one tool over another?
Currently, tool calls are tracked in BaseAgentState.tool_calls and returned in ChatResponse.tool_calls, but there is no server-side logging of the tool execution flow.
Current State
In src/agents/base.py:99-118, _agent_node records tool calls into state but does not log:
def _agent_node(self, state: BaseAgentState) -> dict[str, Any]:
messages = self._prepare_messages(state)
response = self.model_with_tools.invoke(messages)
# Tool calls silently tracked, no logging
tool_calls = state.get("tool_calls", [])
if hasattr(response, "tool_calls") and response.tool_calls:
for tc in response.tool_calls:
tool_calls.append({"name": tc["name"], "args": tc["args"]})
return {"messages": [response], "tool_calls": tool_calls}Proposed Changes
1. Log tool calls in _agent_node
Add logger.info when the model makes tool calls:
INFO agent: Tool call: search_eeglab_docstrings(query="erpimage", limit=5)
INFO agent: Tool call: retrieve_eeglab_docs(title="Plotting ERP images")
2. Log tool results in ToolNode or post-processing
Log success/failure and result length:
INFO agent: Tool result: search_eeglab_docstrings -> 5 results (1234 chars)
INFO agent: Tool result: search_eeglab_docstrings -> "Knowledge base not initialized..."
3. Log fallback/error tool responses
When tools return known error patterns (e.g., "Knowledge base not initialized", "No results found"), log at WARNING level so these stand out in logs.
4. Optional: Debug mode for API responses
Add a ?debug=true query parameter that includes the full tool execution trace in the API response, useful for development and troubleshooting without checking server logs.
Files Involved
src/agents/base.py-_agent_node,_should_use_toolssrc/api/routers/community.py- API response construction- LangFuse integration (
src/core/services/llm.py) is already implemented but optional; this issue is about basic logging that works without LangFuse.