Skip to content

Stream device screen for real-time preview #19

@obj-p

Description

@obj-p

Add a streaming mode for preview rendering that continuously sends screen updates rather than requiring explicit preview_snapshot calls.

Potential approaches

iOS Simulator:

  • Facebook's IDB (idb_companion) supports video streaming from simulators — could be used directly or as reference for the protocol
  • SimDeviceFramebufferService (IOSurface) for direct framebuffer access (related to Direct framebuffer screenshot via IOSurface #11)
  • simctl io recordVideo as a fallback

macOS:

Transport:

  • Server-sent events or streaming MCP response with periodic frame data
  • WebSocket-based frame streaming

This would enable AI agents to observe animations, transitions, and real-time state changes without polling snapshot calls.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions