[Bug] BYOK Anthropic provider missing multiple session events (turn_start, turn_end, reasoning, usage_info)

[lynn-orrell]

Description

When using a BYOK Anthropic provider (type: "anthropic"), the assistant.turn_start and assistant.turn_end events are never emitted. All other events (assistant.usage, assistant.message, tool.execution_start, tool.execution_complete, session.idle) fire correctly.

The same session setup with the default backend or a BYOK OpenAI provider emits all turn lifecycle events as expected.

Reproduction

import asyncio
import os
from copilot import CopilotClient
from copilot.client import SubprocessConfig
from copilot.session import (
    InfiniteSessionConfig,
    PermissionHandler,
    SystemMessageReplaceConfig,
)
from copilot.tools import Tool, ToolResult

async def my_tool(args):
    return ToolResult(content="42")

async def run_test(client, label, model, provider=None):
    events = []
    def on_event(event):
        events.append(event.type.value)

    kwargs = dict(
        on_permission_request=PermissionHandler.approve_all,
        model=model,
        system_message=SystemMessageReplaceConfig(
            mode="replace", content="Use my_tool when asked."
        ),
        available_tools=["my_tool"],
        streaming=True,
        tools=[Tool(name="my_tool", description="Returns 42", handler=my_tool)],
        infinite_sessions=InfiniteSessionConfig(enabled=False),
        on_event=on_event,
    )
    if provider:
        kwargs["provider"] = provider

    session = await client.create_session(**kwargs)
    try:
        await session.send_and_wait("Use my_tool to get the answer.", timeout=60)
    finally:
        await session.disconnect()

    unique = set(events)
    ts = events.count("assistant.turn_start")
    te = events.count("assistant.turn_end")
    print(f"{label}: turn_start={ts}, turn_end={te}, "
          f"usage={events.count('assistant.usage')}, "
          f"message={events.count('assistant.message')}")
    return unique

async def main():
    endpoint = os.environ["AZURE_FOUNDRY_ENDPOINT"].rstrip("/")
    api_key = os.environ["AZURE_FOUNDRY_API_KEY"]
    client = CopilotClient(SubprocessConfig())

    try:
        # 1. Default backend -- turn events fire
        default = await run_test(client, "Default (claude-haiku-4.5)", "claude-haiku-4.5")

        # 2. BYOK Anthropic (Haiku) -- turn events MISSING
        byok_haiku = await run_test(client, "BYOK Anthropic (claude-haiku-4-5)", "claude-haiku-4-5", {
            "type": "anthropic",
            "base_url": f"{endpoint}/anthropic",
            "api_key": api_key,
        })

        # 3. BYOK Anthropic (Sonnet) -- turn events MISSING
        byok_sonnet = await run_test(client, "BYOK Anthropic (claude-sonnet-4-6)", "claude-sonnet-4-6", {
            "type": "anthropic",
            "base_url": f"{endpoint}/anthropic",
            "api_key": api_key,
        })

        # 4. BYOK OpenAI -- turn events fire
        byok_openai = await run_test(client, "BYOK OpenAI (gpt-5.4-nano)", "gpt-5.4-nano", {
            "type": "openai",
            "base_url": f"{endpoint}/openai/v1/",
            "api_key": api_key,
            "wire_api": "responses",
        })

        # Show events emitted by both Default and BYOK OpenAI
        # but never delivered to on_event by BYOK Anthropic
        missing = sorted((default | byok_openai) - byok_haiku)
        if missing:
            print(f"\nEvents missing from BYOK Anthropic that both others emit:")
            for e in missing:
                print(f"  - {e}")
    finally:
        await client.stop()

asyncio.run(main())

Requires AZURE_FOUNDRY_ENDPOINT and AZURE_FOUNDRY_API_KEY env vars pointing to an Azure AI Foundry deployment with Anthropic and OpenAI models available.

Actual output (SDK 0.2.2)

Default (claude-haiku-4.5):              turn_start=2, turn_end=2, usage=2, message=2
BYOK Anthropic (claude-haiku-4-5):       turn_start=0, turn_end=0, usage=2, message=2
BYOK Anthropic (claude-sonnet-4-6):      turn_start=0, turn_end=0, usage=2, message=2
BYOK OpenAI (gpt-5.4-nano):             turn_start=2, turn_end=2, usage=2, message=2

Events missing from BYOK Anthropic that both others emit:
  - assistant.reasoning
  - assistant.reasoning_delta
  - assistant.turn_end
  - assistant.turn_start
  - session.custom_agents_updated
  - session.usage_info

Expected output

All three backends should emit assistant.turn_start and assistant.turn_end. The streaming events documentation lists both as non-ephemeral events that are always emitted, and the agent loop documentation states "the number of assistant.turn_start / assistant.turn_end pairs equals the total number of LLM API calls."

Additional missing events with BYOK Anthropic

Comparing unique event types across all three backends, BYOK Anthropic also does not emit these events that both Default and BYOK OpenAI do emit:

• assistant.reasoning
• assistant.reasoning_delta
• session.usage_info

Note: session.custom_agents_updated is missing from both BYOK providers (Anthropic and OpenAI) but present in the default backend -- this may be a separate, broader BYOK issue.

Impact

Code that tracks per-turn token usage by accumulating assistant.usage data between turn_start/turn_end boundaries gets no per-turn stats with BYOK Anthropic. The assistant.usage events themselves fire correctly, so session-level totals can still be computed, but turn-level granularity is lost.

Environment

• SDK: github-copilot-sdk==0.2.2 (also confirmed on 0.2.1)
• Provider: Azure AI Foundry with type: "anthropic" provider config
• Models tested: claude-haiku-4-5 and claude-sonnet-4-6 via BYOK Anthropic -- both missing turn events
• Controls: Default backend (claude-haiku-4.5) and BYOK OpenAI (gpt-5.4-nano) on the same Foundry endpoint both emit turn events correctly
• OS: Windows 11
• Python: 3.12

[github-actions]

Investigation Findings

Thanks for the detailed reproduction script and comparison data — this made the investigation straightforward.

What I investigated

• python/copilot/client.py — the notification handler that receives session.event JSON-RPC notifications from the CLI and dispatches them via session._dispatch_event(event).
• python/copilot/session.py — the _dispatch_event method that forwards events to registered on_event handlers.
• docs/features/agent-loop.md and docs/features/streaming-events.md — the documented contract for turn lifecycle events.

What I found

This is a confirmed bug. The SDK itself is working correctly — the event dispatch pipeline in _dispatch_event forwards every event it receives from the CLI without any filtering by provider type. There is no code in the SDK that suppresses assistant.turn_start or assistant.turn_end based on which backend is in use.

The root cause is that the Copilot CLI backend is not emitting these events when an Anthropic BYOK provider (type: "anthropic") is configured. When the CLI uses the default backend or an OpenAI BYOK provider, it sends turn lifecycle events through the JSON-RPC notification channel as expected. With Anthropic BYOK, those notifications simply never arrive at the SDK.

This directly contradicts the documented contract:

• docs/features/agent-loop.md: "each iteration of this loop is exactly one LLM API call, visible as one assistant.turn_start / assistant.turn_end pair in the event log"
• docs/features/streaming-events.md: both events are listed as non-ephemeral, always-emitted events

The same gap also explains the other missing events you identified: assistant.reasoning, assistant.reasoning_delta, and session.usage_info are all events that the CLI's Anthropic provider path appears to skip emitting. The session.custom_agents_updated omission for all BYOK providers may be a related but separate issue.

Label applied

bug — the behavior deviates from the documented contract and from the behavior of all other provider configurations.

Generated by Bug Handler for issue #1064 · ● 734.4K · ◷

[authora-aria]

This looks like the Anthropic BYOK adapter is only forwarding content/tool/usage deltas and never mapping the provider lifecycle callbacks into SDK session events. I’d check the type: "anthropic" path for where the stream is normalized and compare it against the OpenAI/provider-agnostic adapter: if assistant.turn_start is emitted when the first assistant chunk arrives and assistant.turn_end when the stream closes (or after final message_stop), the Anthropic adapter probably just needs the same wrapping. Anthropic’s event model usually gives you enough to synthesize these even if there isn’t a 1:1 wire event.

A practical fix is:

• emit assistant.turn_start on first assistant-facing stream event (message_start / first content block / first text delta), guarded by a per-turn boolean
• accumulate reasoning/thinking blocks if Anthropic exposes them separately and forward as assistant.reasoning
• emit assistant.usage/usage_info from the final message_delta/terminal event where token counts are complete
• emit assistant.turn_end in the stream finalizer, not only on a specific provider event, so early-stop/tool paths still close the turn

If you want to catch regressions here, an adapter-level fixture test that asserts event order for anthropic vs openai on the same prompt/tool flow would make this obvious: turn_start -> message/reasoning/tool* -> usage -> turn_end -> session.idle.