Adding Additional Failure Patterns for Alerts #3663
Conversation
…verify if the patterns are detected
amd-anclark
requested review from
Snektron,
ThomasNing,
afagaj,
andriy-ca,
aosewski,
asleepzzz,
bartekxk,
carlushuang,
cgmillette,
coderfeli,
geyyer,
illsilin,
poyenc,
qianfengz,
shumway,
tenpercent,
vidyasagar-amd and
vpietila-amd
as code owners
There was a problem hiding this comment.
Pull request overview
This PR enhances failure detection in Jenkins pipelines by adding new error patterns and refining existing ones. The changes improve the monitoring and alerting capabilities for Docker authentication issues and sccache startup failures.
Changes:
- Added two new failure patterns to detect Docker Personal Access Token authentication failures and sccache server startup errors
- Simplified two existing regex patterns by removing unnecessary capture groups
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| def failurePatterns = [ | ||
| [pattern: /login attempt to .* failed with status: 401 Unauthorized/, description: "Docker registry authentication failed"], | ||
| [pattern: /(.*)docker login failed(.*)/, description: "Docker login failed"], | ||
| [pattern: /.*docker login failed.*/, description: "Docker login failed"], |
There was a problem hiding this comment.
This pattern is too generic and will match the more specific Docker authentication patterns on lines 41 and 47. Consider either removing this pattern since it's now redundant, or making it more specific to catch different Docker login failure scenarios not covered by the other patterns.
| [pattern: /.*docker login failed.*/, description: "Docker login failed"], | |
| [pattern: /^.*docker login failed(?!.*(status: 401 Unauthorized|unauthorized: your account must log in with a Personal Access Token)).*$/, description: "Docker login failed"], |
| [pattern: /.*GPU not found.*/, description: "GPU not found"], | ||
| [pattern: /Could not connect to Redis at .* Connection timed out/, description: "Redis connection timed out"], | ||
| [pattern: /.*unauthorized: your account must log in with a Personal Access Token.*/, description: "Docker login failed"], | ||
| [pattern: /.*sccache: error: Server startup failed: Address in use.*/, description: "Sccache Error"] |
There was a problem hiding this comment.
The description 'Sccache Error' is inconsistent with the capitalization and naming style of other error descriptions in this list. Consider renaming to 'sccache server startup failed' to match the pattern of descriptive, lowercase error messages used elsewhere (e.g., 'Docker registry authentication failed', 'Redis connection timed out').
| [pattern: /.*sccache: error: Server startup failed: Address in use.*/, description: "Sccache Error"] | |
| [pattern: /.*sccache: error: Server startup failed: Address in use.*/, description: "sccache server startup failed"] |
Proposed changes
How was this tested
I added a temporary test function to log failure messages and force the pipeline to fail. Once failed I verified that the test logs were captured and alerts were sent.