Skip to content

Conversation

@kirsten-emak
Copy link

Description

Fixes tool call loss that occurs when Gemini sends tool calls in separate WebSocket messages after signaling turn completion.

When using Gemini 2.5 Flash live native audio 12-2025 with dynamic thinking, tool calls may arrive in separate WebSocket messages after turnComplete. Previously, when turnComplete arrived, all channels including functionChannel were closed and the generation was marked done. If a toolCall then arrived in a subsequent message, it would trigger a new generation with a new functionChannel - but Agent Activity was still reading from the old (now closed) channel, causing the tool call to be lost.

This fix ensures tool calls are written to the correct functionChannel and not lost:

  1. Keep functionChannel open when turnComplete arrives (other channels still close normally)
  2. Only start new generation for serverContent or standalone toolCalls (when functionChannel is already closed)
  3. Close functionChannel in handleToolCall after writing tool calls
  4. Close orphaned functionChannels when starting a new generation (for turns that complete without any tool calls)

Changes Made

  • Modified markCurrentGenerationDone() to accept optional keepFunctionChannelOpen parameter, allowing channels to remain open for late-arriving tool calls
  • Updated onReceiveMessage() to only create new generations for serverContent or standalone toolCall messages (when functionChannel is already closed)
  • Enhanced handleToolCall() to properly close the functionChannel after writing tool calls and detect closed channels
  • Added cleanup in startNewGeneration() to close orphaned functionChannels from previous turns that didn't receive tool calls

Pre-Review Checklist

  • Build passes: All builds (lint, typecheck, tests) pass locally
  • AI-generated code reviewed: Removed unnecessary comments and ensured code quality
  • Changes explained: All changes are properly documented and justified above
  • Scope appropriate: All changes relate to the PR title, or explanations provided for why they're included
  • Video demo: A small video demo showing changes works as expected and did not break any existing functionality using Agent Playground (if applicable) - not applicable

Testing

  • Tested with Gemini 2.5 Flash live native audio 12-2025 with dynamic thinking, proactivity and affective dialog enabled
  • Verified VAD resumes properly after thinking-only generation
  • Tested multiple thinking-heavy and tool calling scenarios to trigger race condition, verified no tool loss (was observed in ~30% of tool-heavy sessions prior to the change)
  • No regression in normal tool call handling
  • Agent properly returns to listening state in all scenarios
  • All tests pass
  • Make sure both restaurant_agent.ts and realtime_agent.ts work properly (for major changes) - specifically related to realtime agent tool handling

Additional Notes


Note to reviewers: Please ensure the pre-review checklist is completed before starting your review.

When using Gemini 2.5 Flash with dynamic thinking, tool calls may arrive in separate WebSocket messages after turnComplete. Previously, when turnComplete arrived, all channels including functionChannel were closed and the generation was marked done. If a toolCall then arrived in a
subsequent message, it would trigger a new generation with a new functionChannel - but Agent Activity was still reading from the old (now closed) channel, causing the tool call to be lost.

This fix ensures tool calls are written to the correct functionChannel and not lost:

1. Keep functionChannel open when turnComplete arrives (other channels still close normally)
2. Only start new generation for serverContent or standalone toolCalls (when functionChannel is already closed)
3. Close functionChannel in handleToolCall after writing tool calls
4. Close orphaned functionChannels when starting a new generation (for turns that complete without any tool calls)
@changeset-bot
Copy link

changeset-bot bot commented Jan 1, 2026

⚠️ No Changeset found

Latest commit: 692aba0

Merging this PR will not cause a version bump for any packages. If these changes should not result in a new version, you're good to go. If these changes should result in a version bump, you need to add a changeset.

This PR includes no changesets

When changesets are added to this PR, you'll see the packages that this PR includes changesets for and the associated semver types

Click here to learn what changesets are, and how to add one.

Click here if you're a maintainer who wants to add a changeset to this PR

Copy link
Contributor

@toubatbrian toubatbrian left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Overall LG! Thanks for the comprehensive testing.

Could you fix errors brought from the CI and add a patch bump to the google plugin in changeset? Thanks!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants