Conversation
Replace simctl subprocess with direct framebuffer capture via CoreSimulator's SimDeviceIOClient for iOS simulator screenshots. Falls back to simctl if IOSurface access fails. Switch both macOS and iOS paths from PNG to JPEG (default quality 0.85) for ~10-50x smaller payloads. Migrate macOS snapshots from deprecated CGWindowListCreateImage to ScreenCaptureKit's SCScreenshotManager. Addresses #11, #26, and partially #19. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
ScreenCaptureKit requires Screen Recording TCC permission, which fails for a headless MCP server. Use NSView.cacheDisplay(in:to:) instead — captures the content view directly from the layer tree with no permissions needed. Also reverts callers to synchronous since cacheDisplay is not async. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- Fix simctl fallback passing --type=jpeg when JPEG was requested, preventing MIME type mismatch (PNG data labeled as JPEG) - Cache CIContext as static singleton instead of recreating per capture - Wrap ioPortDescriptor call in @try/@catch for private API resilience - Inline screenshotViaSimctl into screenshotDataViaSimctl - Clamp quality parameter to 0.0-1.0 (not 1.5) - Update SnapshotCommand to default to .jpg and respect file extension - Remove unnecessary CoreGraphics linked framework - Add PNG output test case (jpegQuality: 1.0) Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
xcrun simctl io screenshotsubprocess (~200-500ms) with direct IOSurface framebuffer capture via CoreSimulator'sSimDeviceIOClient(~5-10ms). Falls back to simctl if IOSurface access fails.qualityparameter topreview_snapshotMCP tool.CGWindowListCreateImagewithNSView.cacheDisplay(in:to:)— captures content view directly, no Screen Recording TCC permission needed.Addresses #11, #26, and partially #19.
Changes
SimulatorBridge.h/m— NewSBCaptureFramebuffer()accessingSimDevice.io→ IO ports →SimDisplayIOSurfaceRenderable→ IOSurface → CIImage → JPEG/PNGSimulatorManager.swift— Tries direct capture first, falls back to simctl subprocessSnapshot.swift—NSView.cacheDisplaywith JPEG/PNG format supportMCPServer.swift—preview_snapshotaccepts optionalqualityparam (0.0–1.0 = JPEG, ≥1.0 = PNG)Package.swift— Link IOSurface, CoreImage, CoreGraphics, ImageIO, UniformTypeIdentifiers on SimulatorBridgeTest plan
swift build— clean build, zero warningsswift test --filter "PreviewParser|IOSHostBuilder|ThunkGenerator|LiteralDiffer|Compiler|BuildSystem"— all 29 fast tests passswift test --filter "bootAndShutdown"— validates IOSurface capture on real booted simulator🤖 Generated with Claude Code