.NET: Make AG-UI conversion API public for multi-agent orchestrations

[darthmolen]

.NET: Make AG-UI conversion API public for multi-agent orchestrations

Target repo: microsoft/agent-framework
Labels: .NET, ag-ui
Related: #2988 (dynamic agent resolution in MapAGUI)

---

Summary

The current AG-UI integration in Microsoft.Agents.AI.Hosting.AGUI.AspNetCore is designed for a single AIAgent processing one request-response stream via MapAGUI. All of the conversion machinery (BaseEvent, AsAGUIEventStreamAsync, AGUIServerSentEventsResult, the per-event types) is internal, which means consumers building multi-agent orchestrations cannot reuse the SDK's AG-UI conversion logic and have to reimplement the protocol themselves.

We're asking for these conversion utilities to be made public (or for an equivalent public API) so that orchestration frameworks can merge per-agent streams and orchestration-level domain events into a single AG-UI SSE stream.

This is a sibling issue to #2988 -- both are blocked by the same internal types, but from a different angle: #2988 needs them for dynamic agent resolution at request time, we need them for merging streams from multiple agents that don't share a single AIAgent lifecycle.

Background: our use case

We're building a multi-agent swarm orchestrator on top of Microsoft.Extensions.AI. A swarm is a long-running background task with this lifecycle:

POST /api/swarm           SwarmManager.CreateSwarmAsync(goal)   →  fire-and-forget
                                              │
                                              ▼
                                  SwarmOrchestrator.RunAsync()
                                              │
            ┌─────────────────────────────────┼─────────────────────────────────┐
            ▼                                 ▼                                 ▼
        Leader Agent                    Worker Agent 1                    Worker Agent N
        (IChatClient)                   (IChatClient)                     (IChatClient)
        create_plan                     task_update                       inbox_send
        submit_report                   inbox_send                        task_update

GET /api/swarm/{id}/stream    ←   Single SSE stream multiplexing events from ALL agents
                                  + orchestration-level events (phases, tasks, rounds)

Key characteristics:

1. N agents per swarm. Each SwarmAgent wraps its own IChatClient (with FunctionInvokingChatClient). The leader agent runs first to plan, then N worker agents run concurrently across multiple rounds, then the leader runs again for synthesis.

2. Fire-and-forget lifecycle. The swarm is started by a POST and runs as a background task in a BackgroundService. Clients connect to a GET /api/swarm/{id}/stream SSE endpoint at any point and receive a STATE_SNAPSHOT followed by live events. There is no single request-response that maps to a single agent run.

3. Two event sources merged into one stream:

   • Per-agent events -- tool calls, text messages emitted as agents do their work. These naturally fit the AG-UI protocol's TOOL_CALL_* and TEXT_MESSAGE_* event types.
   • Orchestration events -- phase transitions, task creation, agent spawning, inter-agent inbox messages, round numbers, swarm completion. These don't come from any single agent's ChatResponse -- they're emitted by the orchestrator.

4. Late-joining clients need a snapshot. We use AG-UI's STATE_SNAPSHOT and STATE_DELTA (RFC 6902 JSON Patch) to hydrate the UI when a client connects mid-execution.

We want to use AG-UI as the wire protocol because of CopilotKit compatibility and because it's a clean, standardized event format. We don't want to invent yet another event protocol.

Why MapAGUI doesn't fit

MapAGUI("/", agent) is a perfect match for one agent, one request, one response stream:

endpoints.MapPost(pattern, async ([FromBody] RunAgentInput? input, ...) =>
{
    var events = aiAgent.RunStreamingAsync(messages, options, ct)
        .AsChatResponseUpdatesAsync()
        .FilterServerToolsFromMixedToolInvocationsAsync(...)
        .AsAGUIEventStreamAsync(input.ThreadId, input.RunId, jsonSerializerOptions, ct);
    return new AGUIServerSentEventsResult(events, sseLogger);
});

But this doesn't fit our model:

• We don't have a single AIAgent to call RunStreamingAsync on. We have N agents managed by an orchestrator.
• We don't have a single runId for the whole swarm -- each agent's interaction is part of a larger orchestration with its own phase model.
• We can't return events as the response to a POST because the swarm is started by a different request than the one streaming events.
• We need to inject orchestration-level events that don't originate from any agent's ChatResponse.

What we want to be able to write

// Our orchestrator emits to a Channel<BaseEvent>
public sealed class SwarmEventAdapter
{
    private readonly Channel<BaseEvent> channel;

    public ChannelReader<BaseEvent> Reader => this.channel.Reader;

    public Task EmitAgentEventsAsync(IChatClient agentClient, ChatResponse response)
    {
        // Use the SDK's public conversion to turn agent ChatResponse into BaseEvents
        foreach (var evt in ChatResponseUpdateAGUIExtensions.ConvertToAGUIEvents(
            response, agentName, jsonOptions))
        {
            this.channel.Writer.TryWrite(evt);
        }
        return Task.CompletedTask;
    }

    public Task EmitOrchestrationEventAsync(BaseEvent evt) =>
        this.channel.Writer.WriteAsync(evt).AsTask();
}

// Our long-running SSE endpoint subscribes to the adapter
endpoints.MapGet("/api/swarm/{id}/stream", async (Guid id, ISwarmManager mgr, HttpContext ctx) =>
{
    var execution = mgr.GetSwarm(id);
    return new AGUIServerSentEventsResult(execution.Adapter.Reader.ReadAllAsync(), logger);
});

The two things we need from the SDK to make this possible:

1. Public BaseEvent hierarchy -- the per-event types (RunStartedEvent, RunFinishedEvent, RunErrorEvent, StepStartedEvent, StepFinishedEvent, TextMessage*Event, ToolCall*Event, StateSnapshotEvent, StateDeltaEvent) so we can construct events directly for orchestration-level things.

2. Public AGUIServerSentEventsResult (or an IResult factory equivalent) so we can return an SSE stream from a custom endpoint, with the SDK handling the SSE framing, JSON serialization, and error event emission.

A bonus would be a public conversion helper:

public static IEnumerable<BaseEvent> ToAGUIEvents(
    this ChatResponse response,
    string threadId,
    string runId,
    JsonSerializerOptions jsonOptions);

So that for each agent's ChatResponse, we can get back the equivalent AG-UI events without reimplementing the FunctionCallContent → TOOL_CALL_* and TextContent → TEXT_MESSAGE_* mapping ourselves.

What we did instead (the workaround)

Because the SDK types are internal, we built a parallel implementation:

• Our own public SwarmAgUIEvent hierarchy with the same JSON wire format as the SDK's internal types (using JsonPolymorphic discriminators)
• Our own DelegatingChatClient interceptor (AgUIEventInterceptor) that converts ChatResponse content into our event types
• Our own SSE writer (SseEventWriter.FormatAgUIEvent) that serializes via System.Text.Json
• Our own channel-based event hub (SwarmEventAdapter) that the orchestrator emits to and the SSE endpoint reads from

This works, but:

• We're shipping a parallel implementation of the same protocol the SDK already implements
• We have to keep our wire format in sync with any AG-UI spec changes the SDK absorbs
• We can't benefit from any new features (e.g. STEP_STARTED was missing from AGUIEventTypes so we had to add it)
• We don't get to participate in the SDK's testing of the protocol

Proposed solution

Make these types public:

• BaseEvent and all RunXxxEvent, StepXxxEvent, TextMessageXxxEvent, ToolCallXxxEvent, StateXxxEvent derived types
• AGUIServerSentEventsResult (or provide an IResult factory)
• ChatResponseUpdateAGUIExtensions.AsAGUIEventStreamAsync() and AsChatResponseUpdatesAsync()
• AGUIEventTypes constants

Or equivalently: provide a small public surface that exposes the same capability behind a different API. Concretely the two methods we need are:

namespace Microsoft.Agents.AI.Hosting.AGUI.AspNetCore;

public static class AGUIEventStreamExtensions
{
    /// <summary>
    /// Converts a stream of ChatResponseUpdate into AG-UI BaseEvents,
    /// wrapping the stream with RunStarted/RunFinished events.
    /// </summary>
    public static IAsyncEnumerable<BaseEvent> AsAGUIEventStreamAsync(
        this IAsyncEnumerable<ChatResponseUpdate> updates,
        string threadId,
        string runId,
        JsonSerializerOptions jsonSerializerOptions,
        CancellationToken cancellationToken = default);
}

public static class AGUIResults
{
    /// <summary>
    /// Returns an IResult that streams AG-UI events as SSE.
    /// </summary>
    public static IResult ServerSentEvents(IAsyncEnumerable<BaseEvent> events);
}

Plus making BaseEvent and the concrete event types public so consumers can construct them.

Compatibility

Making types public is backward-compatible -- no existing code breaks. Consumers who use MapAGUI continue to work exactly as today. The new APIs are additive.

If there's hesitation about exposing the full event hierarchy, an alternative is a "builder" API:

public sealed class AGUIEventBuilder
{
    public BaseEvent RunStarted(string threadId, string runId);
    public BaseEvent StepStarted(string stepName);
    public BaseEvent StateSnapshot(JsonElement snapshot);
    public BaseEvent StateDelta(JsonElement jsonPatch);
    // ... etc
}

This hides the concrete types but gives consumers the ability to construct events.

We're happy to contribute

We have a working parallel implementation in our own codebase that proves the multi-agent use case works once you have the right primitives. We'd be glad to:

1. Open a draft PR that:

   • Promotes the relevant types to public
   • Adds an AGUIResults.ServerSentEvents(IAsyncEnumerable<BaseEvent>) factory
   • Adds tests demonstrating multi-agent merging into a single SSE stream
   • Adds a sample showing a multi-agent orchestrator using the new public API

2. Maintain the public surface area in line with whatever conventions the maintainers prefer (e.g. [EditorBrowsable(Never)] on event types if you want to discourage direct construction in favor of the builder API, etc.)

Just let us know which direction the maintainers prefer and we'll put together a PR.

Environment

• Microsoft.Agents.AI.Hosting.AGUI.AspNetCore (current preview)
• Microsoft.Extensions.AI 10.3.0
• .NET 10
• Use case: multi-agent swarm orchestrator with per-swarm SSE streams