Skip to content

Update Claude build analysis prompt and model to Sonnet 4.6#22760

Open
iangmaia wants to merge 4 commits intotrunkfrom
iangmaia/update-claude-build-analysis
Open

Update Claude build analysis prompt and model to Sonnet 4.6#22760
iangmaia wants to merge 4 commits intotrunkfrom
iangmaia/update-claude-build-analysis

Conversation

@iangmaia
Copy link
Copy Markdown
Contributor

@iangmaia iangmaia commented Apr 2, 2026

Description

  • Migrate Claude build analysis model from Sonnet 4.5 to Sonnet 4.6
  • Switch to iangmaia/claude-summarize fork with build_log_mode: "failed" and max_log_lines: 1500 — only feeds failed job logs to Claude for more focused analysis
  • Gate Claude analysis on real failures: upload-claude-analysis.sh checks non-essential step outcomes (currently Danger) via buildkite-agent before uploading the pipeline. When only Danger failed, Claude is skipped entirely — no annotation, no PR comment
  • Simplify the custom prompt now that non-essential jobs are filtered upstream

Test Steps

  • Verify the Claude analysis step passes (green) and does not post a PR comment when only Danger fails

🤖 Generated with Claude Code

Improve the custom prompt to ignore non-real failures (the intentional
exit 1 trigger, Danger PR Check, and broken/skipped jobs) and respond
briefly when no actual failures exist. Migrate model from Sonnet 4.5
to Sonnet 4.6.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
@iangmaia iangmaia self-assigned this Apr 2, 2026
Use iangmaia/claude-summarize fork which adds build_log_mode, max_log_lines,
and on-failure trigger. This feeds only failed job logs to Claude (capped at
1500 lines) for more focused analysis with less noise.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
@wpmobilebot
Copy link
Copy Markdown
Contributor

wpmobilebot commented Apr 2, 2026

App Icon📲 You can test the changes from this Pull Request in Jetpack Android by scanning the QR code below to install the corresponding build.

App NameJetpack Android
Build TypeDebug
Versionpr22760-a64c849
Build Number1488
Application IDcom.jetpack.android.prealpha
Commita64c849
Installation URL7q9kholqaha4o
Automatticians: You can use our internal self-serve MC tool to give yourself access to those builds if needed.

@wpmobilebot
Copy link
Copy Markdown
Contributor

wpmobilebot commented Apr 2, 2026

App Icon📲 You can test the changes from this Pull Request in WordPress Android by scanning the QR code below to install the corresponding build.

App NameWordPress Android
Build TypeDebug
Versionpr22760-a64c849
Build Number1488
Application IDorg.wordpress.android.prealpha
Commita64c849
Installation URL5ucqdbohee5d0
Automatticians: You can use our internal self-serve MC tool to give yourself access to those builds if needed.

Add upload-claude-analysis.sh that checks non-essential step outcomes
before uploading the Claude analysis pipeline. When only Danger or
other non-critical jobs failed, the pipeline is not uploaded — no
analysis, no annotation, no PR comment.

Also simplify the custom prompt now that Danger is filtered upstream.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
@iangmaia iangmaia added this to the 26.8 milestone Apr 2, 2026
@iangmaia iangmaia requested review from mokagio and twstokes April 2, 2026 18:58
@iangmaia iangmaia enabled auto-merge (squash) April 2, 2026 19:01
When real failures co-occur with a Danger failure, Claude should still
ignore Danger rather than wasting analysis space on it.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
@sonarqubecloud
Copy link
Copy Markdown

sonarqubecloud bot commented Apr 2, 2026

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants