Skip to content

BidiAgent: Support Nova Sonic v2 async tool calling #2021

@dminhk

Description

@dminhk

Summary

Nova Sonic v2 supports asynchronous tool calling where the model continues conversing naturally while tools execute in the background (docs). The current BidiAgent implementation executes tools synchronously — blocking the conversation until toolResult is sent back.

Current Behavior

  1. Nova Sonic sends toolUse event
  2. BidiAgent intercepts it, executes the Python tool function synchronously
  3. BidiAgent sends toolResult back to Nova Sonic
  4. Nova Sonic can only resume speaking after receiving the result

During step 2-3, the conversation is paused. For fast tools (<1s) this is fine, but for tools that take 5-30s (API calls, database queries, web search), the user experiences silence and Nova Sonic may timeout (55s idle limit → error 532).

Desired Behavior

  1. Nova Sonic sends toolUse event
  2. BidiAgent starts tool execution in a background task
  3. BidiAgent does not block — Nova Sonic keeps talking ("Let me check that for you...")
  4. When tool completes, BidiAgent sends toolResult asynchronously
  5. Nova Sonic incorporates the result into the ongoing conversation

This matches the async tool calling pattern documented by AWS:

"Unlike traditional synchronous tool calling where the AI waits silently for tool results, Amazon Nova 2 Sonic's asynchronous approach allows it to continue accepting user input while tools are running, respond to new questions without waiting for pending tool results, and maintain natural conversation flow without awkward pauses."

Use Case

Voice agents that delegate tasks to external services (API calls, agent-to-agent delegation, database lookups) need the conversation to remain interactive during tool execution. With async tool calling:

  • Voice says "Let me check that for you" (immediately)
  • User can ask other questions while waiting
  • When results arrive, voice naturally incorporates them

Workaround

Currently the options are:

  1. Design tools to return instantly (fire-and-forget) and feed results back as text input later
  2. Auto-reconnect with chat history when Nova Sonic times out during long tool calls
  3. Keep tools under ~1s execution time

These work but are fragile — the session may drop and reconnect instead of staying alive.

Environment

  • strands-agents[bidi] >= 1.33.0
  • amazon.nova-2-sonic-v1:0
  • Python 3.12

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions