[fix](pipeline) fix comment-trigger workflow crash due to GitHub API rate limit#63304
Open
hello-stephen wants to merge 1 commit into
Open
[fix](pipeline) fix comment-trigger workflow crash due to GitHub API rate limit#63304hello-stephen wants to merge 1 commit into
hello-stephen wants to merge 1 commit into
Conversation
…te limit causes missing all_files
Two issues fixed:
1. `_get_pr_changed_files_count` and `_get_pr_changed_files` in github-utils.sh
made unauthenticated curl requests, hitting GitHub's 60 req/h anonymous rate
limit on shared Actions runner IPs. Added `${GITHUB_TOKEN:+-H "Authorization:
Bearer ${GITHUB_TOKEN}"}` so requests use the 5000 req/h authenticated limit
when GITHUB_TOKEN is available (no-op when unset, preserving local usage).
2. When `_get_pr_changed_files` fails, step 4 falls back to trigger-all but does
NOT create the `all_files` file. Step 5 then crashes with "No such file or
directory" on `< all_files`, causing conclusion=failure and skipping all
TeamCity trigger steps. Added an early-exit guard in step 5 to handle this
gracefully.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Contributor
Author
|
Thank you for your contribution to Apache Doris. Please clearly describe your PR:
|
Contributor
Author
|
run buildall |
Contributor
Author
TPC-H: Total hot run time: 30898 ms |
Contributor
Author
TPC-DS: Total hot run time: 168894 ms |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
When a PR author posts a second
run buildallcomment within a short window, thecomment-to-trigger-teamcityworkflow can silently fail, leaving TeamCitynot triggered even though the Actions run appears to have executed.
Root cause is two connected bugs:
Bug 1 — Unauthenticated GitHub API calls hit rate limits
_get_pr_changed_files_countand_get_pr_changed_filesinregression-test/pipeline/common/github-utils.shmake anonymouscurlrequests(no
Authorizationheader). GitHub's anonymous rate limit is 60 req/h per IP.GitHub Actions runner IPs are shared across every workflow in the org, so hitting
the limit is common when many workflows run concurrently.
Bug 2 — Missing
all_filescrashes step 5When
_get_pr_changed_filesfails (10 retries exhausted), step 4 ("Check if prneed run build") has a correct fallback that defaults everything to trigger-all —
but it does not create the
all_filesfile.Step 5 ("Check for sensitive pipeline script changes") then executes:
This makes step 5 fail, which causes all downstream TeamCity trigger steps to be
skipped. The workflow shows
conclusion: failurewith no useful message to the PRauthor.
Fix
regression-test/pipeline/common/github-utils.sh— add optional auth headerto both curl calls using the shell idiom
${GITHUB_TOKEN:+-H "Authorization: Bearer ${GITHUB_TOKEN}"}. WhenGITHUB_TOKENis set (always the case in GitHubActions), the authenticated rate limit of 5 000 req/h is used. When unset
(local manual usage), the flag expands to nothing and behaviour is unchanged.
.github/workflows/comment-to-trigger-teamcity.yml— add an early-exit guardin step 5 before reading
all_files. If the file is absent (because the API callin step 4 failed), the step exits 0 with an explanatory warning instead of
crashing. The previous step already defaulted to trigger-all in this scenario, so
no functionality is lost.
Reproduction
Observed on run 25771630114
triggered by PR #63110 comment #63110 (comment).