ref: Unify rendering pipeline with structured output and snapshot tests

AGENTS.md

@@ -46,6 +46,7 @@ Use these sections under `## [Unreleased]`:
 - Append to existing subsections (e.g., `### Fixed`), do not create duplicates
 - NEVER modify already-released version sections (e.g., `## [0.12.2]`)
 - Each version section is immutable once released
+- NEVER update snapshot fixtures unless asked to do so, these are integration tests, on failure assume code is wrong before questioning the fixture
 -
 #### Attribution
 - **Internal changes (from issues)**: `Fixed foo bar ([#123](https://github.com/cameroncook/XcodeBuildMCP/issues/123))`

benchmarks/simulator-test/2026-03-17T14-18-19-390Z/flowdeck-run-1.stdout.txt

@@ -0,0 +1,51 @@
+^DUsing scheme test configuration: Debug
+🧪 Test: CalculatorApp
+Workspace: /Users/cameroncooke/.codex/worktrees/43f4/XcodeBuildMCP/example_projects/iOS_Calculator/CalculatorApp.xcworkspace
+Configuration: Debug
+Target: iPhone 17 Pro
+🧪 Finding available tests...
+Resolved to 21 test(s):
+- CalculatorAppTests/CalculatorAppTests/testAppLaunch
+- CalculatorAppTests/CalculatorAppTests/testCalculationPerformance
+- CalculatorAppTests/CalculatorAppTests/testCalculatorOperationsEnum
+- CalculatorAppTests/CalculatorAppTests/testCalculatorServiceBasicOperation
+- CalculatorAppTests/CalculatorAppTests/testCalculatorServiceChainedOperations
+... and 16 more
+🧪 Running tests (0, 0 failures)
+🧪 Running tests (1, 0 failures)
+🧪 Running tests (2, 0 failures)
+🧪 Running tests (3, 0 failures)
+🧪 Running tests (4, 0 failures)
+🧪 Running tests (5, 0 failures)
+🧪 Running tests (6, 0 failures)
+🧪 Running tests (7, 0 failures)/Users/cameroncooke/.codex/worktrees/43f4/XcodeBuildMCP/example_projects/iOS_Calculator/CalculatorAppTests/CalculatorAppTests.swift:52: error: -[CalculatorAppTests.CalculatorAppTests testCalculatorServiceFailure] : XCTAssertEqual failed: ("0") is not equal to ("999") - This test should fail - display should be 0, not 999
+🧪 Running tests (8, 1 failure)
+🧪 Running tests (9, 1 failure)
+🧪 Running tests (10, 1 failure)
+🧪 Running tests (11, 1 failure)
+🧪 Running tests (12, 1 failure)
+🧪 Running tests (13, 1 failure)
+🧪 Running tests (14, 1 failure)
+🧪 Running tests (15, 1 failure)
+🧪 Running tests (16, 1 failure)
+🧪 Running tests (17, 1 failure)
+🧪 Running tests (18, 1 failure)
+🧪 Running tests (19, 1 failure)
+🧪 Running tests (20, 1 failure)
+🧪 Running tests (21, 1 failure)
+Failed Tests
+CalculatorAppTests
+✗ testCalculatorServiceFailure (0.003s)
+└─ XCTAssertEqual failed: ("0") is not
+equal to ("999") - This test should fail
+- display should be 0, not 999
+(CalculatorAppTests.swift:52)
+Test Summary
+╭───────────────────────────────────────╮
+│ Total:    21                          │
+│ Passed:   20                          │
+│ Failed:   1                           │
+│ Skipped:  0                           │
+│ Duration: 27.96s                      │
+╰───────────────────────────────────────╯
+✗ 1 test(s) failed.

benchmarks/simulator-test/2026-03-17T14-18-19-390Z/flowdeck-run-2.stdout.txt

@@ -0,0 +1,51 @@
+^DUsing scheme test configuration: Debug
+🧪 Test: CalculatorApp
+Workspace: /Users/cameroncooke/.codex/worktrees/43f4/XcodeBuildMCP/example_projects/iOS_Calculator/CalculatorApp.xcworkspace
+Configuration: Debug
+Target: iPhone 17 Pro
+🧪 Finding available tests...
+Resolved to 21 test(s):
+- CalculatorAppTests/CalculatorAppTests/testAppLaunch
+- CalculatorAppTests/CalculatorAppTests/testCalculationPerformance
+- CalculatorAppTests/CalculatorAppTests/testCalculatorOperationsEnum
+- CalculatorAppTests/CalculatorAppTests/testCalculatorServiceBasicOperation
+- CalculatorAppTests/CalculatorAppTests/testCalculatorServiceChainedOperations
+... and 16 more
+🧪 Running tests (0, 0 failures)
+🧪 Running tests (1, 0 failures)
+🧪 Running tests (2, 0 failures)
+🧪 Running tests (3, 0 failures)
+🧪 Running tests (4, 0 failures)
+🧪 Running tests (5, 0 failures)
+🧪 Running tests (6, 0 failures)
+🧪 Running tests (7, 0 failures)/Users/cameroncooke/.codex/worktrees/43f4/XcodeBuildMCP/example_projects/iOS_Calculator/CalculatorAppTests/CalculatorAppTests.swift:52: error: -[CalculatorAppTests.CalculatorAppTests testCalculatorServiceFailure] : XCTAssertEqual failed: ("0") is not equal to ("999") - This test should fail - display should be 0, not 999
+🧪 Running tests (8, 1 failure)
+🧪 Running tests (9, 1 failure)
+🧪 Running tests (10, 1 failure)
+🧪 Running tests (11, 1 failure)
+🧪 Running tests (12, 1 failure)
+🧪 Running tests (13, 1 failure)
+🧪 Running tests (14, 1 failure)
+🧪 Running tests (15, 1 failure)
+🧪 Running tests (16, 1 failure)
+🧪 Running tests (17, 1 failure)
+🧪 Running tests (18, 1 failure)
+🧪 Running tests (19, 1 failure)
+🧪 Running tests (20, 1 failure)
+🧪 Running tests (21, 1 failure)
+Failed Tests
+CalculatorAppTests
+✗ testCalculatorServiceFailure (0.009s)
+└─ XCTAssertEqual failed: ("0") is not
+equal to ("999") - This test should fail
+- display should be 0, not 999
+(CalculatorAppTests.swift:52)
+Test Summary
+╭───────────────────────────────────────╮
+│ Total:    21                          │
+│ Passed:   20                          │
+│ Failed:   1                           │
+│ Skipped:  0                           │
+│ Duration: 20.31s                      │
+╰───────────────────────────────────────╯
+✗ 1 test(s) failed.

benchmarks/simulator-test/2026-03-17T14-18-19-390Z/flowdeck-run-3.stdout.txt

@@ -0,0 +1,51 @@
+^DUsing scheme test configuration: Debug
+🧪 Test: CalculatorApp
+Workspace: /Users/cameroncooke/.codex/worktrees/43f4/XcodeBuildMCP/example_projects/iOS_Calculator/CalculatorApp.xcworkspace
+Configuration: Debug
+Target: iPhone 17 Pro
+🧪 Finding available tests...
+Resolved to 21 test(s):
+- CalculatorAppTests/CalculatorAppTests/testAppLaunch
+- CalculatorAppTests/CalculatorAppTests/testCalculationPerformance
+- CalculatorAppTests/CalculatorAppTests/testCalculatorOperationsEnum
+- CalculatorAppTests/CalculatorAppTests/testCalculatorServiceBasicOperation
+- CalculatorAppTests/CalculatorAppTests/testCalculatorServiceChainedOperations
+... and 16 more
+🧪 Running tests (0, 0 failures)
+🧪 Running tests (1, 0 failures)
+🧪 Running tests (2, 0 failures)
+🧪 Running tests (3, 0 failures)
+🧪 Running tests (4, 0 failures)
+🧪 Running tests (5, 0 failures)
+🧪 Running tests (6, 0 failures)
+🧪 Running tests (7, 0 failures)/Users/cameroncooke/.codex/worktrees/43f4/XcodeBuildMCP/example_projects/iOS_Calculator/CalculatorAppTests/CalculatorAppTests.swift:52: error: -[CalculatorAppTests.CalculatorAppTests testCalculatorServiceFailure] : XCTAssertEqual failed: ("0") is not equal to ("999") - This test should fail - display should be 0, not 999
+🧪 Running tests (8, 1 failure)
+🧪 Running tests (9, 1 failure)
+🧪 Running tests (10, 1 failure)
+🧪 Running tests (11, 1 failure)
+🧪 Running tests (12, 1 failure)
+🧪 Running tests (13, 1 failure)
+🧪 Running tests (14, 1 failure)
+🧪 Running tests (15, 1 failure)
+🧪 Running tests (16, 1 failure)
+🧪 Running tests (17, 1 failure)
+🧪 Running tests (18, 1 failure)
+🧪 Running tests (19, 1 failure)
+🧪 Running tests (20, 1 failure)
+🧪 Running tests (21, 1 failure)
+Failed Tests
+CalculatorAppTests
+✗ testCalculatorServiceFailure (0.009s)
+└─ XCTAssertEqual failed: ("0") is not
+equal to ("999") - This test should fail
+- display should be 0, not 999
+(CalculatorAppTests.swift:52)
+Test Summary
+╭───────────────────────────────────────╮
+│ Total:    21                          │
+│ Passed:   20                          │
+│ Failed:   1                           │
+│ Skipped:  0                           │
+│ Duration: 16.44s                      │
+╰───────────────────────────────────────╯
+✗ 1 test(s) failed.

benchmarks/simulator-test/2026-03-17T14-18-19-390Z/summary.json

@@ -0,0 +1,74 @@
+{
+  "generatedAt": "2026-03-17T14:20:36.285Z",
+  "mode": "warm",
+  "iterations": 3,
+  "workspacePath": "/Users/cameroncooke/.codex/worktrees/43f4/XcodeBuildMCP/example_projects/iOS_Calculator/CalculatorApp.xcworkspace",
+  "results": [
+    {
+      "tool": "xcodebuildmcp",
+      "iteration": 1,
+      "exitCode": 1,
+      "wallClockMs": 29067.379917000002,
+      "firstStdoutMs": 2.148291999999998,
+      "firstMilestoneMs": 2152.612708,
+      "startupToFirstStreamedTestProgressMs": 13020.933500000001,
+      "stdoutPath": "/Users/cameroncooke/.codex/worktrees/43f4/XcodeBuildMCP/benchmarks/simulator-test/2026-03-17T14-18-19-390Z/xcodebuildmcp-run-1.stdout.txt",
+      "stderrPath": "/Users/cameroncooke/.codex/worktrees/43f4/XcodeBuildMCP/benchmarks/simulator-test/2026-03-17T14-18-19-390Z/xcodebuildmcp-run-1.stderr.txt"
+    },
+    {
+      "tool": "flowdeck",
+      "iteration": 1,
+      "exitCode": 1,
+      "wallClockMs": 28296.29575,
+      "firstStdoutMs": 3.5727919999990263,
+      "firstMilestoneMs": 12480.404625,
+      "startupToFirstStreamedTestProgressMs": 12480.409000000003,
+      "stdoutPath": "/Users/cameroncooke/.codex/worktrees/43f4/XcodeBuildMCP/benchmarks/simulator-test/2026-03-17T14-18-19-390Z/flowdeck-run-1.stdout.txt",
+      "stderrPath": "/Users/cameroncooke/.codex/worktrees/43f4/XcodeBuildMCP/benchmarks/simulator-test/2026-03-17T14-18-19-390Z/flowdeck-run-1.stderr.txt"
+    },
+    {
+      "tool": "xcodebuildmcp",
+      "iteration": 2,
+      "exitCode": 1,
+      "wallClockMs": 20358.631999999998,
+      "firstStdoutMs": 3.855666999996174,
+      "firstMilestoneMs": 1894.4525829999984,
+      "startupToFirstStreamedTestProgressMs": 6474.262499999997,
+      "stdoutPath": "/Users/cameroncooke/.codex/worktrees/43f4/XcodeBuildMCP/benchmarks/simulator-test/2026-03-17T14-18-19-390Z/xcodebuildmcp-run-2.stdout.txt",
+      "stderrPath": "/Users/cameroncooke/.codex/worktrees/43f4/XcodeBuildMCP/benchmarks/simulator-test/2026-03-17T14-18-19-390Z/xcodebuildmcp-run-2.stderr.txt"
+    },
+    {
+      "tool": "flowdeck",
+      "iteration": 2,
+      "exitCode": 1,
+      "wallClockMs": 20567.050875,
+      "firstStdoutMs": 3.934166000006371,
+      "firstMilestoneMs": 5885.525833000007,
+      "startupToFirstStreamedTestProgressMs": 5885.5267500000045,
+      "stdoutPath": "/Users/cameroncooke/.codex/worktrees/43f4/XcodeBuildMCP/benchmarks/simulator-test/2026-03-17T14-18-19-390Z/flowdeck-run-2.stdout.txt",
+      "stderrPath": "/Users/cameroncooke/.codex/worktrees/43f4/XcodeBuildMCP/benchmarks/simulator-test/2026-03-17T14-18-19-390Z/flowdeck-run-2.stderr.txt"
+    },
+    {
+      "tool": "xcodebuildmcp",
+      "iteration": 3,
+      "exitCode": 1,
+      "wallClockMs": 21910.729708,
+      "firstStdoutMs": 3.3832499999989523,
+      "firstMilestoneMs": 2140.4143329999933,
+      "startupToFirstStreamedTestProgressMs": 6239.000874999998,
+      "stdoutPath": "/Users/cameroncooke/.codex/worktrees/43f4/XcodeBuildMCP/benchmarks/simulator-test/2026-03-17T14-18-19-390Z/xcodebuildmcp-run-3.stdout.txt",
+      "stderrPath": "/Users/cameroncooke/.codex/worktrees/43f4/XcodeBuildMCP/benchmarks/simulator-test/2026-03-17T14-18-19-390Z/xcodebuildmcp-run-3.stderr.txt"
+    },
+    {
+      "tool": "flowdeck",
+      "iteration": 3,
+      "exitCode": 1,
+      "wallClockMs": 16693.48666699999,
+      "firstStdoutMs": 3.411791999998968,
+      "firstMilestoneMs": 5152.938666999995,
+      "startupToFirstStreamedTestProgressMs": 5152.9394579999935,
+      "stdoutPath": "/Users/cameroncooke/.codex/worktrees/43f4/XcodeBuildMCP/benchmarks/simulator-test/2026-03-17T14-18-19-390Z/flowdeck-run-3.stdout.txt",
+      "stderrPath": "/Users/cameroncooke/.codex/worktrees/43f4/XcodeBuildMCP/benchmarks/simulator-test/2026-03-17T14-18-19-390Z/flowdeck-run-3.stderr.txt"
+    }
+  ]
+}