-
Notifications
You must be signed in to change notification settings - Fork 757
BidiAgent: Support Nova Sonic v2 async tool calling #2021
Description
Summary
Nova Sonic v2 supports asynchronous tool calling where the model continues conversing naturally while tools execute in the background (docs). The current BidiAgent implementation executes tools synchronously — blocking the conversation until toolResult is sent back.
Current Behavior
- Nova Sonic sends
toolUseevent - BidiAgent intercepts it, executes the Python tool function synchronously
- BidiAgent sends
toolResultback to Nova Sonic - Nova Sonic can only resume speaking after receiving the result
During step 2-3, the conversation is paused. For fast tools (<1s) this is fine, but for tools that take 5-30s (API calls, database queries, web search), the user experiences silence and Nova Sonic may timeout (55s idle limit → error 532).
Desired Behavior
- Nova Sonic sends
toolUseevent - BidiAgent starts tool execution in a background task
- BidiAgent does not block — Nova Sonic keeps talking ("Let me check that for you...")
- When tool completes, BidiAgent sends
toolResultasynchronously - Nova Sonic incorporates the result into the ongoing conversation
This matches the async tool calling pattern documented by AWS:
"Unlike traditional synchronous tool calling where the AI waits silently for tool results, Amazon Nova 2 Sonic's asynchronous approach allows it to continue accepting user input while tools are running, respond to new questions without waiting for pending tool results, and maintain natural conversation flow without awkward pauses."
Use Case
Voice agents that delegate tasks to external services (API calls, agent-to-agent delegation, database lookups) need the conversation to remain interactive during tool execution. With async tool calling:
- Voice says "Let me check that for you" (immediately)
- User can ask other questions while waiting
- When results arrive, voice naturally incorporates them
Workaround
Currently the options are:
- Design tools to return instantly (fire-and-forget) and feed results back as text input later
- Auto-reconnect with chat history when Nova Sonic times out during long tool calls
- Keep tools under ~1s execution time
These work but are fragile — the session may drop and reconnect instead of staying alive.
Environment
strands-agents[bidi]>= 1.33.0amazon.nova-2-sonic-v1:0- Python 3.12