Skip to content

NO-ISSUE: Add 4.21 and 4.22 jobs for lvm-operator#79040

Open
pacevedom wants to merge 1 commit intoopenshift:mainfrom
pacevedom:lvm-4.x-jobs
Open

NO-ISSUE: Add 4.21 and 4.22 jobs for lvm-operator#79040
pacevedom wants to merge 1 commit intoopenshift:mainfrom
pacevedom:lvm-4.x-jobs

Conversation

@pacevedom
Copy link
Copy Markdown
Contributor

@pacevedom pacevedom commented May 8, 2026

Summary

This pull request adds OpenShift CI infrastructure support for LVM Operator releases targeting OpenShift 4.21 and 4.22.

LVM Operator CI Configuration

Two new CI operator configuration files are introduced for the LVM Operator:

  • openshift-lvm-operator-release-4.21.yaml — Defines the complete CI/CD pipeline for the 4.21 release branch, including:

    • Build images (pre-commit image, main operator image, must-gather image, integration tests)
    • Test suite covering pre-commit checks, unit tests, and E2E tests across AWS (single-node, standard, hypershift, ARM variants) and bare metal (dualstack SNO/MNO configurations)
    • Operator bundle generation and promotion to the lvms namespace
    • Resource requirements and release channel definitions
  • openshift-lvm-operator-release-4.22.yaml — Mirrors the 4.21 configuration structure for the 4.22 release branch with equivalent build artifacts, test pipelines, and deployment targets

Generated Prow job definitions are created for both versions, covering presubmit validation, postsubmit testing, and periodic scheduled runs.

MicroShift Infrastructure Enhancement

The MicroShift AWS RPM installation step was updated to enforce the exact Go toolchain version specified in the project's go.mod when REBASE_TO is set. The step now:

  • Reads the required Go version from the microshift repository
  • Downloads and installs the matching Go toolchain if a mismatch is detected
  • Proceeds with Python bootstrapping and dependency installation

The corresponding step reference was updated to source from a specific base image (rhel-9-release-golang-1.25-openshift-4.23) instead of a generic root image.

@openshift-ci-robot openshift-ci-robot added the jira/valid-reference Indicates that this PR references a valid Jira ticket of any type. label May 8, 2026
@openshift-ci-robot
Copy link
Copy Markdown
Contributor

@pacevedom: This pull request explicitly references no jira issue.

Details

In response to this:

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

@coderabbitai
Copy link
Copy Markdown
Contributor

coderabbitai Bot commented May 8, 2026

Warning

Rate limit exceeded

@pacevedom has exceeded the limit for the number of commits that can be reviewed per hour. Please wait 41 minutes and 44 seconds before requesting another review.

You’ve run out of usage credits. Purchase more in the billing tab.

⌛ How to resolve this issue?

After the wait time has elapsed, a review can be triggered using the @coderabbitai review command as a PR comment. Alternatively, push new commits to this PR.

We recommend that you space out your commits to avoid hitting the rate limit.

🚦 How do rate limits work?

CodeRabbit enforces hourly rate limits for each developer per organization.

Our paid plans have higher rate limits than the trial, open-source and free plans. In all cases, we re-allow further reviews after a brief timeout.

Please see our FAQ for further information.

ℹ️ Review info
⚙️ Run configuration

Configuration used: Repository YAML (base), Central YAML (inherited)

Review profile: CHILL

Plan: Enterprise

Run ID: 2540088f-f5f4-46ce-ad18-1a8eb9ed4de9

📥 Commits

Reviewing files that changed from the base of the PR and between 2417398 and f8e6129.

⛔ Files ignored due to path filters (6)
  • ci-operator/jobs/openshift/lvm-operator/openshift-lvm-operator-release-4.21-periodics.yaml is excluded by !ci-operator/jobs/**
  • ci-operator/jobs/openshift/lvm-operator/openshift-lvm-operator-release-4.21-postsubmits.yaml is excluded by !ci-operator/jobs/**
  • ci-operator/jobs/openshift/lvm-operator/openshift-lvm-operator-release-4.21-presubmits.yaml is excluded by !ci-operator/jobs/**
  • ci-operator/jobs/openshift/lvm-operator/openshift-lvm-operator-release-4.22-periodics.yaml is excluded by !ci-operator/jobs/**
  • ci-operator/jobs/openshift/lvm-operator/openshift-lvm-operator-release-4.22-postsubmits.yaml is excluded by !ci-operator/jobs/**
  • ci-operator/jobs/openshift/lvm-operator/openshift-lvm-operator-release-4.22-presubmits.yaml is excluded by !ci-operator/jobs/**
📒 Files selected for processing (2)
  • ci-operator/config/openshift/lvm-operator/openshift-lvm-operator-release-4.21.yaml
  • ci-operator/config/openshift/lvm-operator/openshift-lvm-operator-release-4.22.yaml

Walkthrough

Two new OpenShift LVM operator release CI configurations (versions 4.21 and 4.22) are added with complete build, test, and promotion pipelines. A separate microshift infrastructure script is updated to enforce matching Go toolchain versions during builds, with its corresponding step registry reference updated to use a specific release image.

Changes

LVM Operator Release Configurations (4.21 & 4.22)

Layer / File(s) Summary
Base Images & Build Root
ci-operator/config/openshift/lvm-operator/openshift-lvm-operator-release-4.21.yaml, openshift-lvm-operator-release-4.22.yaml
Base images defined (cli-operator-sdk, dev-scripts, must-gather) with OCP RHEL9 release images for 4.21 and 4.22 respectively.
Build Images & Operator Bundle
ci-operator/config/openshift/lvm-operator/openshift-lvm-operator-release-4.2*.yaml
Build graph establishes pre-commit image (with Snyk CLI), operator image, must-gather image, and integration-test image. Operator bundle configured via bundle.Dockerfile with pipeline substitution references.
Promotion & Release Channels
ci-operator/config/openshift/lvm-operator/openshift-lvm-operator-release-4.2*.yaml
Promotion rules target lvms namespace with version-specific tags (v4.21, v4.22). Release metadata pins initial, latest, and multi-latest candidate streams for each version.
Resource Limits & Pre-Commit Tests
ci-operator/config/openshift/lvm-operator/openshift-lvm-operator-release-4.2*.yaml
Global and profile-specific resource requests/limits established. Pre-commit, unit, and postsubmit unit test stages configured with Codecov secret mounting for coverage collection.
Snyk Security & E2E Tests
ci-operator/config/openshift/lvm-operator/openshift-lvm-operator-release-4.2*.yaml
Optional Snyk code and dependency scan tests defined. Multiple E2E/integration jobs for AWS (single-node, standard, hypershift, SNO/MNO QE) and Equinix metal dualstack variants, each with cluster profiles, workflows, and JUnit output collection.
Generated Metadata
ci-operator/config/openshift/lvm-operator/openshift-lvm-operator-release-4.2*.yaml
Metadata pins branch to corresponding release-4.21 and release-4.22, org to openshift, and repo to lvm-operator.

MicroShift RPM Install AWS Go Toolchain Enforcement

Layer / File(s) Summary
Base Image Reference
ci-operator/step-registry/openshift/microshift/infra/rpm-install-aws/openshift-microshift-infra-rpm-install-aws-ref.yaml
from_image updated to reference release image in openshift namespace with tag rhel-9-release-golang-1.25-openshift-4.23 instead of from: root.
Go Toolchain Version Management
ci-operator/step-registry/openshift/microshift/infra/rpm-install-aws/openshift-microshift-infra-rpm-install-aws-commands.sh
When REBASE_TO is set, script now reads required Go version from microshift's go.mod, compares to installed version, and downloads/extracts matching Go release if mismatch detected, updating GOROOT and PATH. Python bootstrap and package installation remain but execute after Go setup.

Estimated code review effort

🎯 2 (Simple) | ⏱️ ~12 minutes

🚥 Pre-merge checks | ✅ 12
✅ Passed checks (12 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title accurately describes the main change: adding CI jobs for lvm-operator versions 4.21 and 4.22.
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
Stable And Deterministic Test Names ✅ Passed This PR modifies CI configuration files (YAML) and a bash script, not Ginkgo test files. The check for stable/deterministic Ginkgo test names is not applicable to non-test code.
Test Structure And Quality ✅ Passed Check is not applicable. The PR contains CI configuration files and infrastructure scripts, not Ginkgo test code. The test structure and quality check is designed for test files.
Microshift Test Compatibility ✅ Passed This PR does not add any Ginkgo e2e tests. It only modifies CI infrastructure files (YAML configurations and bash scripts). The custom check for MicroShift test compatibility is not applicable.
Single Node Openshift (Sno) Test Compatibility ✅ Passed PR does not add new Ginkgo e2e tests. Changes are limited to CI configuration YAML files and bash infrastructure setup scripts, which are out of scope for the SNO test compatibility check.
Topology-Aware Scheduling Compatibility ✅ Passed PR adds CI configuration and infrastructure setup scripts, not deployment manifests or operator code. No scheduling constraints or topology-dependent logic are introduced to customer clusters.
Ote Binary Stdout Contract ✅ Passed The OTE Binary Stdout Contract check is not applicable. The PR modifies only CI configuration YAML files and a bash infrastructure script—no Go test binaries or code subject to this contract.
Ipv6 And Disconnected Network Test Compatibility ✅ Passed PR does not introduce new Ginkgo e2e test code files. It only adds CI configuration (YAML) and infrastructure setup (bash script) files. Custom check is not applicable.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

@pacevedom
Copy link
Copy Markdown
Contributor Author

/pj-rehearse ?

@openshift-merge-bot
Copy link
Copy Markdown
Contributor

@pacevedom: now processing your pj-rehearse request. Please allow up to 10 minutes for jobs to trigger or cancel.

@openshift-ci openshift-ci Bot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label May 8, 2026
@openshift-ci openshift-ci Bot requested review from jaypoulz and suleymanakbas91 May 8, 2026 10:48
@openshift-merge-bot
Copy link
Copy Markdown
Contributor

@pacevedom: job(s): ? either don't exist or were not found to be affected, and cannot be rehearsed

@pacevedom
Copy link
Copy Markdown
Contributor Author

/pj-rehearse max

@openshift-merge-bot
Copy link
Copy Markdown
Contributor

@pacevedom: now processing your pj-rehearse request. Please allow up to 10 minutes for jobs to trigger or cancel.

@pacevedom
Copy link
Copy Markdown
Contributor Author

/pj-rehearse max

@openshift-merge-bot
Copy link
Copy Markdown
Contributor

@pacevedom: now processing your pj-rehearse request. Please allow up to 10 minutes for jobs to trigger or cancel.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Copy link
Copy Markdown
Contributor

@coderabbitai coderabbitai Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 6

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In
`@ci-operator/config/openshift/lvm-operator/openshift-lvm-operator-release-4.21.yaml`:
- Around line 47-50: The Dockerfile uses a non-deterministic Snyk download (curl
.../cli/latest/snyk-linux) without integrity verification; update the RUN steps
that use SNYK_DIR to download a pinned Snyk release (use an explicit versioned
URL instead of "latest") and add a checksum verification step (e.g., download
the corresponding .sha256 or .sha256sum and verify with sha256sum -c or compare
the computed checksum) before running chmod and adding SNYK_DIR to PATH so the
build fails on checksum mismatch.
- Around line 400-428: Add a timeout: 4h0m0s to the baremetal integration test
steps that currently lack it: insert timeout: 4h0m0s under the test step that
contains the lvms-sno-integration-test (part of the
e2e-baremetalds-sno-dualstack-qe-integration-tests job) and likewise under the
test step that contains lvms-mno-integration-test (part of the
e2e-baremetalds-mno-dualstack-qe-integration-tests job) so both test blocks have
the same 4-hour cutoff as the AWS QE jobs.

In
`@ci-operator/config/openshift/lvm-operator/openshift-lvm-operator-release-4.22.yaml`:
- Around line 47-50: The Dockerfile currently downloads the SNYK CLI from a
"latest" URL without integrity checks (the RUN curl .../snyk-linux and ENV PATH
lines referencing SNYK_DIR); change this to pin a specific Snyk release (use a
versioned URL) and add a verification step: download the corresponding checksum
(or signature) for that version, verify the binary's SHA256 (or GPG) before
chmod +x, and fail the build if verification fails; keep using ${SNYK_DIR} and
the PATH ENV but ensure the steps explicitly use curl -fSL (or similar), compare
the checksum with sha256sum, and remove the binary on mismatch so the build
cannot proceed with an unverified artifact.
- Around line 400-428: The baremetal QE integration test steps are missing a
timeout guard; add timeout: 4h0m0s to the test job definitions that run
"./integration-test run-suite..."—specifically the job blocks containing the
test entries with as: lvms-sno-integration-test and as:
lvms-mno-integration-test (the
e2e-baremetalds-sno/mno-dualstack-qe-integration-tests workflow entries) so each
of those test steps includes timeout: 4h0m0s at the same level as
cli/from/resources to prevent hung runners from consuming clusters indefinitely.

In
`@ci-operator/step-registry/openshift/microshift/infra/rpm-install-aws/openshift-microshift-infra-rpm-install-aws-commands.sh`:
- Around line 74-95: The toolchain/bootstrap block that installs Go and Python
tools is missing its REBASE_TO guard because the opening if for REBASE_TO was
commented out; restore the conditional so the commands that reference
required_go, current_go, GOROOT, and pip3 (the block around the grep for go.mod,
the go install steps, and python3 -m ensurepip / pip3 install) only run when
REBASE_TO is set. Concretely, re-enable the leading "if [ -n \"${REBASE_TO}\" ];
then" that wraps the toolchain/bootstrap steps (matching the existing closing
"if [ -n \"${REBASE_TO}\" ]; then" at the end) so non-rebase jobs skip the
Go/Python bootstrap.
- Line 84: Update the curl invocation that downloads the Go tarball (the line
containing curl -sSfL "https://go.dev/dl/go${required_go}.linux-amd64.tar.gz" -o
/tmp/go.tar.gz) to include retry and timeout controls such as --retry (e.g. 5),
--retry-delay (e.g. 5), --connect-timeout (e.g. 10) and --max-time (e.g. 300) so
transient network failures are retried and the request doesn’t hang; keep
existing flags (-sSfL) and output destination unchanged.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Repository YAML (base), Central YAML (inherited)

Review profile: CHILL

Plan: Enterprise

Run ID: 7686f16e-c8e6-43fe-9da9-6bd52779fd14

📥 Commits

Reviewing files that changed from the base of the PR and between f0d000e and 2417398.

⛔ Files ignored due to path filters (6)
  • ci-operator/jobs/openshift/lvm-operator/openshift-lvm-operator-release-4.21-periodics.yaml is excluded by !ci-operator/jobs/**
  • ci-operator/jobs/openshift/lvm-operator/openshift-lvm-operator-release-4.21-postsubmits.yaml is excluded by !ci-operator/jobs/**
  • ci-operator/jobs/openshift/lvm-operator/openshift-lvm-operator-release-4.21-presubmits.yaml is excluded by !ci-operator/jobs/**
  • ci-operator/jobs/openshift/lvm-operator/openshift-lvm-operator-release-4.22-periodics.yaml is excluded by !ci-operator/jobs/**
  • ci-operator/jobs/openshift/lvm-operator/openshift-lvm-operator-release-4.22-postsubmits.yaml is excluded by !ci-operator/jobs/**
  • ci-operator/jobs/openshift/lvm-operator/openshift-lvm-operator-release-4.22-presubmits.yaml is excluded by !ci-operator/jobs/**
📒 Files selected for processing (4)
  • ci-operator/config/openshift/lvm-operator/openshift-lvm-operator-release-4.21.yaml
  • ci-operator/config/openshift/lvm-operator/openshift-lvm-operator-release-4.22.yaml
  • ci-operator/step-registry/openshift/microshift/infra/rpm-install-aws/openshift-microshift-infra-rpm-install-aws-commands.sh
  • ci-operator/step-registry/openshift/microshift/infra/rpm-install-aws/openshift-microshift-infra-rpm-install-aws-ref.yaml

Comment on lines +47 to +50
RUN mkdir -p ${SNYK_DIR}
RUN curl https://static.snyk.io/cli/latest/snyk-linux -o ${SNYK_DIR}/snyk
RUN chmod +x ${SNYK_DIR}/snyk
ENV PATH=$PATH:${SNYK_DIR}
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟡 Minor | ⚡ Quick win

Same Snyk curl supply-chain concern as in the 4.22 config — pin the version and verify the checksum.

curl .../cli/latest/snyk-linux is non-deterministic and lacks an integrity check. See the parallel comment on the 4.22 file for the suggested fix pattern.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In
`@ci-operator/config/openshift/lvm-operator/openshift-lvm-operator-release-4.21.yaml`
around lines 47 - 50, The Dockerfile uses a non-deterministic Snyk download
(curl .../cli/latest/snyk-linux) without integrity verification; update the RUN
steps that use SNYK_DIR to download a pinned Snyk release (use an explicit
versioned URL instead of "latest") and add a checksum verification step (e.g.,
download the corresponding .sha256 or .sha256sum and verify with sha256sum -c or
compare the computed checksum) before running chmod and adding SNYK_DIR to PATH
so the build fails on checksum mismatch.

Comment on lines +400 to +428
test:
- as: lvms-sno-integration-test
cli: latest
commands: |
./integration-test run-suite -c 1 --junit-path ${ARTIFACT_DIR}/junit_results.xml openshift/lvm-operator/test/integration/qe_tests/sno
from: lvm-operator-integration-test
resources:
requests:
cpu: 100m
memory: 200Mi
workflow: cucushift-installer-rehearse-baremetalds-ipi-ovn-dualstack-sno-lvms
- as: e2e-baremetalds-mno-dualstack-qe-integration-tests
cron: '@weekly'
steps:
cluster_profile: equinix-ocp-metal-qe
env:
LVM_OPERATOR_SUB_INSTALL_NAMESPACE: openshift-lvm-storage
LVM_OPERATOR_SUB_SOURCE: lvm-catalogsource
test:
- as: lvms-mno-integration-test
cli: latest
commands: |
./integration-test run-suite -c 1 --junit-path ${ARTIFACT_DIR}/junit_results.xml openshift/lvm-operator/test/integration/qe_tests/mno
from: lvm-operator-integration-test
resources:
requests:
cpu: 100m
memory: 200Mi
workflow: cucushift-installer-rehearse-baremetalds-ipi-ovn-dualstack-mno-lvms
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟡 Minor | ⚡ Quick win

Both baremetal QE integration test steps are missing timeout: 4h0m0s.

e2e-baremetalds-sno-dualstack-qe-integration-tests and e2e-baremetalds-mno-dualstack-qe-integration-tests run the same -c 1 --junit-path integration suite as the AWS QE jobs (all of which carry an explicit 4-hour timeout), but the baremetal steps omit it. On expensive bare-metal hardware a hung test step would block the cluster with no automatic cutoff.

Apply the same timeout: 4h0m0s fix shown in the 4.22 comment to both baremetal test steps here.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In
`@ci-operator/config/openshift/lvm-operator/openshift-lvm-operator-release-4.21.yaml`
around lines 400 - 428, Add a timeout: 4h0m0s to the baremetal integration test
steps that currently lack it: insert timeout: 4h0m0s under the test step that
contains the lvms-sno-integration-test (part of the
e2e-baremetalds-sno-dualstack-qe-integration-tests job) and likewise under the
test step that contains lvms-mno-integration-test (part of the
e2e-baremetalds-mno-dualstack-qe-integration-tests job) so both test blocks have
the same 4-hour cutoff as the AWS QE jobs.

Comment on lines +47 to +50
RUN mkdir -p ${SNYK_DIR}
RUN curl https://static.snyk.io/cli/latest/snyk-linux -o ${SNYK_DIR}/snyk
RUN chmod +x ${SNYK_DIR}/snyk
ENV PATH=$PATH:${SNYK_DIR}
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟡 Minor | ⚡ Quick win

Pin the Snyk CLI to a specific version and verify its integrity.

curl .../snyk-linux pins to latest at image-build time, making the image non-deterministic across rebuilds. More importantly, the downloaded binary is not integrity-verified, which is a supply-chain risk — a compromised or tampered binary would silently execute inside CI.

🛡️ Suggested hardening
-      RUN curl https://static.snyk.io/cli/latest/snyk-linux -o ${SNYK_DIR}/snyk
-      RUN chmod +x ${SNYK_DIR}/snyk
+      # Pin to a known release; update SNYK_VERSION + SNYK_SHA256 when upgrading.
+      ENV SNYK_VERSION=v1.1292.1
+      RUN curl -fsSL https://static.snyk.io/cli/${SNYK_VERSION}/snyk-linux -o ${SNYK_DIR}/snyk \
+          && echo "<expected-sha256>  ${SNYK_DIR}/snyk" | sha256sum -c - \
+          && chmod +x ${SNYK_DIR}/snyk
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In
`@ci-operator/config/openshift/lvm-operator/openshift-lvm-operator-release-4.22.yaml`
around lines 47 - 50, The Dockerfile currently downloads the SNYK CLI from a
"latest" URL without integrity checks (the RUN curl .../snyk-linux and ENV PATH
lines referencing SNYK_DIR); change this to pin a specific Snyk release (use a
versioned URL) and add a verification step: download the corresponding checksum
(or signature) for that version, verify the binary's SHA256 (or GPG) before
chmod +x, and fail the build if verification fails; keep using ${SNYK_DIR} and
the PATH ENV but ensure the steps explicitly use curl -fSL (or similar), compare
the checksum with sha256sum, and remove the binary on mismatch so the build
cannot proceed with an unverified artifact.

Comment on lines +400 to +428
test:
- as: lvms-sno-integration-test
cli: latest
commands: |
./integration-test run-suite -c 1 --junit-path ${ARTIFACT_DIR}/junit_results.xml openshift/lvm-operator/test/integration/qe_tests/sno
from: lvm-operator-integration-test
resources:
requests:
cpu: 100m
memory: 200Mi
workflow: cucushift-installer-rehearse-baremetalds-ipi-ovn-dualstack-sno-lvms
- as: e2e-baremetalds-mno-dualstack-qe-integration-tests
cron: '@weekly'
steps:
cluster_profile: equinix-ocp-metal-qe
env:
LVM_OPERATOR_SUB_INSTALL_NAMESPACE: openshift-lvm-storage
LVM_OPERATOR_SUB_SOURCE: lvm-catalogsource
test:
- as: lvms-mno-integration-test
cli: latest
commands: |
./integration-test run-suite -c 1 --junit-path ${ARTIFACT_DIR}/junit_results.xml openshift/lvm-operator/test/integration/qe_tests/mno
from: lvm-operator-integration-test
resources:
requests:
cpu: 100m
memory: 200Mi
workflow: cucushift-installer-rehearse-baremetalds-ipi-ovn-dualstack-mno-lvms
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟡 Minor | ⚡ Quick win

Add timeout to the baremetal QE integration test steps.

Both e2e-baremetalds-sno-dualstack-qe-integration-tests and e2e-baremetalds-mno-dualstack-qe-integration-tests run the same ./integration-test run-suite -c 1 --junit-path ... command used by the AWS QE variants, all of which explicitly carry timeout: 4h0m0s. The baremetal tests omit that guard, so a hung runner would consume the cluster indefinitely.

⏱️ Proposed fix
 - as: e2e-baremetalds-sno-dualstack-qe-integration-tests
   ...
     - as: lvms-sno-integration-test
       ...
       resources:
         requests:
           cpu: 100m
           memory: 200Mi
+      timeout: 4h0m0s
   workflow: cucushift-installer-rehearse-baremetalds-ipi-ovn-dualstack-sno-lvms
 - as: e2e-baremetalds-mno-dualstack-qe-integration-tests
   ...
     - as: lvms-mno-integration-test
       ...
       resources:
         requests:
           cpu: 100m
           memory: 200Mi
+      timeout: 4h0m0s
   workflow: cucushift-installer-rehearse-baremetalds-ipi-ovn-dualstack-mno-lvms
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In
`@ci-operator/config/openshift/lvm-operator/openshift-lvm-operator-release-4.22.yaml`
around lines 400 - 428, The baremetal QE integration test steps are missing a
timeout guard; add timeout: 4h0m0s to the test job definitions that run
"./integration-test run-suite..."—specifically the job blocks containing the
test entries with as: lvms-sno-integration-test and as:
lvms-mno-integration-test (the
e2e-baremetalds-sno/mno-dualstack-qe-integration-tests workflow entries) so each
of those test steps includes timeout: 4h0m0s at the same level as
cli/from/resources to prevent hung runners from consuming clusters indefinitely.

Comment on lines +74 to +95
#if [ -n "${REBASE_TO}" ]; then
# Under this condition we need to force traps at the last moment to not override the one above.
echo "REBASE_TO is set to ${REBASE_TO}"
export PATH="${HOME}/.local/bin:${PATH}"

# Install the exact Go version required by microshift's go.mod
required_go=$(grep '^go ' /go/src/github.com/openshift/microshift/go.mod | awk '{print $2}')
current_go=$(go version 2>/dev/null | awk '{print $3}' | sed 's/^go//')
if [[ "${current_go}" != "${required_go}" ]]; then
echo "Go version mismatch: have ${current_go:-none}, need ${required_go}. Installing..."
curl -sSfL "https://go.dev/dl/go${required_go}.linux-amd64.tar.gz" -o /tmp/go.tar.gz
mkdir -p /tmp/goroot
tar -C /tmp/goroot -xzf /tmp/go.tar.gz
rm /tmp/go.tar.gz
export GOROOT=/tmp/goroot/go
export PATH="${GOROOT}/bin:${PATH}"
echo "Installed Go $(go version)"
fi

python3 -m ensurepip --upgrade
pip3 install setuptools-rust cryptography pyyaml pygithub gitpython

if [ -n "${REBASE_TO}" ]; then
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟠 Major | ⚡ Quick win

Restore the REBASE_TO guard for the toolchain/bootstrap block.

Line 74 is commented out, so Lines 75-94 run even when REBASE_TO is empty. That makes non-rebase jobs perform Go/Python bootstrap unnecessarily and increases failure surface.

💡 Proposed fix
-#if [ -n "${REBASE_TO}" ]; then
+if [ -n "${REBASE_TO}" ]; then
   # Under this condition we need to force traps at the last moment to not override the one above.
   echo "REBASE_TO is set to ${REBASE_TO}"
   export PATH="${HOME}/.local/bin:${PATH}"

   # Install the exact Go version required by microshift's go.mod
   required_go=$(grep '^go ' /go/src/github.com/openshift/microshift/go.mod | awk '{print $2}')
   current_go=$(go version 2>/dev/null | awk '{print $3}' | sed 's/^go//')
   if [[ "${current_go}" != "${required_go}" ]]; then
     echo "Go version mismatch: have ${current_go:-none}, need ${required_go}. Installing..."
     curl -sSfL "https://go.dev/dl/go${required_go}.linux-amd64.tar.gz" -o /tmp/go.tar.gz
     mkdir -p /tmp/goroot
     tar -C /tmp/goroot -xzf /tmp/go.tar.gz
     rm /tmp/go.tar.gz
     export GOROOT=/tmp/goroot/go
     export PATH="${GOROOT}/bin:${PATH}"
     echo "Installed Go $(go version)"
   fi

   python3 -m ensurepip --upgrade
   pip3 install setuptools-rust cryptography pyyaml pygithub gitpython
-if [ -n "${REBASE_TO}" ]; then
   cp "${CLUSTER_PROFILE_DIR}"/pull-secret "${HOME}"/.pull-secret.json
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In
`@ci-operator/step-registry/openshift/microshift/infra/rpm-install-aws/openshift-microshift-infra-rpm-install-aws-commands.sh`
around lines 74 - 95, The toolchain/bootstrap block that installs Go and Python
tools is missing its REBASE_TO guard because the opening if for REBASE_TO was
commented out; restore the conditional so the commands that reference
required_go, current_go, GOROOT, and pip3 (the block around the grep for go.mod,
the go install steps, and python3 -m ensurepip / pip3 install) only run when
REBASE_TO is set. Concretely, re-enable the leading "if [ -n \"${REBASE_TO}\" ];
then" that wraps the toolchain/bootstrap steps (matching the existing closing
"if [ -n \"${REBASE_TO}\" ]; then" at the end) so non-rebase jobs skip the
Go/Python bootstrap.

current_go=$(go version 2>/dev/null | awk '{print $3}' | sed 's/^go//')
if [[ "${current_go}" != "${required_go}" ]]; then
echo "Go version mismatch: have ${current_go:-none}, need ${required_go}. Installing..."
curl -sSfL "https://go.dev/dl/go${required_go}.linux-amd64.tar.gz" -o /tmp/go.tar.gz
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟠 Major | ⚡ Quick win

🧩 Analysis chain

🏁 Script executed:

#!/bin/bash
# Verify the current download path and whether timeout/retry flags are missing.
rg -n -C2 'go\.dev/dl/go\$\{required_go\}\.linux-amd64\.tar\.gz|curl -sSfL' \
  ci-operator/step-registry/openshift/microshift/infra/rpm-install-aws/openshift-microshift-infra-rpm-install-aws-commands.sh

Repository: openshift/release

Length of output: 379


🏁 Script executed:

#!/bin/bash
# Check for other curl commands in this script to see if they use retry/timeout
rg -n 'curl' ci-operator/step-registry/openshift/microshift/infra/rpm-install-aws/openshift-microshift-infra-rpm-install-aws-commands.sh

Repository: openshift/release

Length of output: 153


🏁 Script executed:

#!/bin/bash
# Search for other Go tarball downloads in the codebase to see common patterns
rg -n 'go\.dev/dl/go.*\.tar\.gz|curl.*--retry' --type sh -A2 -B2 | head -100

Repository: openshift/release

Length of output: 13248


Add retry/timeout controls to the Go tarball download to prevent transient network failures.

The download at line 84 uses only -sSfL flags without retry or timeout controls, making it susceptible to transient network issues that cause avoidable CI flakes. This pattern is inconsistent with other external downloads in the codebase, which commonly include --retry, --retry-delay, --connect-timeout, and --max-time flags.

💡 Proposed fix
-    curl -sSfL "https://go.dev/dl/go${required_go}.linux-amd64.tar.gz" -o /tmp/go.tar.gz
+    curl -sSfL \
+      --retry 5 \
+      --retry-all-errors \
+      --connect-timeout 10 \
+      --max-time 300 \
+      "https://go.dev/dl/go${required_go}.linux-amd64.tar.gz" \
+      -o /tmp/go.tar.gz
📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
curl -sSfL "https://go.dev/dl/go${required_go}.linux-amd64.tar.gz" -o /tmp/go.tar.gz
curl -sSfL \
--retry 5 \
--retry-all-errors \
--connect-timeout 10 \
--max-time 300 \
"https://go.dev/dl/go${required_go}.linux-amd64.tar.gz" \
-o /tmp/go.tar.gz
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In
`@ci-operator/step-registry/openshift/microshift/infra/rpm-install-aws/openshift-microshift-infra-rpm-install-aws-commands.sh`
at line 84, Update the curl invocation that downloads the Go tarball (the line
containing curl -sSfL "https://go.dev/dl/go${required_go}.linux-amd64.tar.gz" -o
/tmp/go.tar.gz) to include retry and timeout controls such as --retry (e.g. 5),
--retry-delay (e.g. 5), --connect-timeout (e.g. 10) and --max-time (e.g. 300) so
transient network failures are retried and the request doesn’t hang; keep
existing flags (-sSfL) and output destination unchanged.

@openshift-merge-bot
Copy link
Copy Markdown
Contributor

[REHEARSALNOTIFIER]
@pacevedom: the pj-rehearse plugin accommodates running rehearsal tests for the changes in this PR. Expand 'Interacting with pj-rehearse' for usage details. The following rehearsable tests have been affected by this change:

Test name Repo Type Reason
pull-ci-openshift-lvm-operator-release-4.21-ci-index-lvm-operator-bundle openshift/lvm-operator presubmit Presubmit changed
pull-ci-openshift-lvm-operator-release-4.21-e2e-aws openshift/lvm-operator presubmit Presubmit changed
pull-ci-openshift-lvm-operator-release-4.21-e2e-aws-hypershift openshift/lvm-operator presubmit Presubmit changed
pull-ci-openshift-lvm-operator-release-4.21-e2e-aws-single-node openshift/lvm-operator presubmit Presubmit changed
pull-ci-openshift-lvm-operator-release-4.21-e2e-aws-single-node-integration-tests openshift/lvm-operator presubmit Presubmit changed
pull-ci-openshift-lvm-operator-release-4.21-images openshift/lvm-operator presubmit Presubmit changed
pull-ci-openshift-lvm-operator-release-4.21-precommit-check openshift/lvm-operator presubmit Presubmit changed
pull-ci-openshift-lvm-operator-release-4.21-snyk-code openshift/lvm-operator presubmit Presubmit changed
pull-ci-openshift-lvm-operator-release-4.21-snyk-deps openshift/lvm-operator presubmit Presubmit changed
pull-ci-openshift-lvm-operator-release-4.21-unit-test openshift/lvm-operator presubmit Presubmit changed
pull-ci-openshift-lvm-operator-release-4.22-ci-index-lvm-operator-bundle openshift/lvm-operator presubmit Presubmit changed
pull-ci-openshift-lvm-operator-release-4.22-e2e-aws openshift/lvm-operator presubmit Presubmit changed
pull-ci-openshift-lvm-operator-release-4.22-e2e-aws-hypershift openshift/lvm-operator presubmit Presubmit changed
pull-ci-openshift-lvm-operator-release-4.22-e2e-aws-single-node openshift/lvm-operator presubmit Presubmit changed
pull-ci-openshift-lvm-operator-release-4.22-e2e-aws-single-node-integration-tests openshift/lvm-operator presubmit Presubmit changed
pull-ci-openshift-lvm-operator-release-4.22-images openshift/lvm-operator presubmit Presubmit changed
pull-ci-openshift-lvm-operator-release-4.22-precommit-check openshift/lvm-operator presubmit Presubmit changed
pull-ci-openshift-lvm-operator-release-4.22-snyk-code openshift/lvm-operator presubmit Presubmit changed
pull-ci-openshift-lvm-operator-release-4.22-snyk-deps openshift/lvm-operator presubmit Presubmit changed
pull-ci-openshift-lvm-operator-release-4.22-unit-test openshift/lvm-operator presubmit Presubmit changed
periodic-ci-openshift-lvm-operator-release-4.21-e2e-aws N/A periodic Periodic changed
periodic-ci-openshift-lvm-operator-release-4.22-e2e-aws-mno-qe-integration-tests N/A periodic Periodic changed
periodic-ci-openshift-lvm-operator-release-4.21-e2e-aws-single-node N/A periodic Periodic changed
periodic-ci-openshift-lvm-operator-release-4.22-e2e-aws-single-node N/A periodic Periodic changed
periodic-ci-openshift-lvm-operator-release-4.21-e2e-aws-mno-qe-integration-tests N/A periodic Periodic changed

A total of 38 jobs have been affected by this change. The above listing is non-exhaustive and limited to 25 jobs.

A full list of affected jobs can be found here

Interacting with pj-rehearse

Comment: /pj-rehearse to run up to 5 rehearsals
Comment: /pj-rehearse skip to opt-out of rehearsals
Comment: /pj-rehearse {test-name}, with each test separated by a space, to run one or more specific rehearsals
Comment: /pj-rehearse more to run up to 10 rehearsals
Comment: /pj-rehearse max to run up to 25 rehearsals
Comment: /pj-rehearse auto-ack to run up to 5 rehearsals, and add the rehearsals-ack label on success
Comment: /pj-rehearse list to get an up-to-date list of affected jobs
Comment: /pj-rehearse abort to abort all active rehearsals
Comment: /pj-rehearse network-access-allowed to allow rehearsals of tests that have the restrict_network_access field set to false. This must be executed by an openshift org member who is not the PR author

Once you are satisfied with the results of the rehearsals, comment: /pj-rehearse ack to unblock merge. When the rehearsals-ack label is present on your PR, merge will no longer be blocked by rehearsals.
If you would like the rehearsals-ack label removed, comment: /pj-rehearse reject to re-block merging.

@pacevedom
Copy link
Copy Markdown
Contributor Author

/pj-rehearse max

@openshift-merge-bot
Copy link
Copy Markdown
Contributor

@pacevedom: now processing your pj-rehearse request. Please allow up to 10 minutes for jobs to trigger or cancel.

@suleymanakbas91
Copy link
Copy Markdown
Contributor

/lgtm

@openshift-ci openshift-ci Bot added the lgtm Indicates that a PR is ready to be merged. label May 8, 2026
@openshift-ci
Copy link
Copy Markdown
Contributor

openshift-ci Bot commented May 8, 2026

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: pacevedom, suleymanakbas91

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Details Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@openshift-ci
Copy link
Copy Markdown
Contributor

openshift-ci Bot commented May 8, 2026

@pacevedom: The following tests failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:

Test name Commit Details Required Rerun command
ci/rehearse/openshift/lvm-operator/release-4.22/ci-index-lvm-operator-bundle f8e6129 link unknown /pj-rehearse pull-ci-openshift-lvm-operator-release-4.22-ci-index-lvm-operator-bundle
ci/rehearse/openshift/lvm-operator/release-4.21/e2e-aws-single-node-integration-tests f8e6129 link unknown /pj-rehearse pull-ci-openshift-lvm-operator-release-4.21-e2e-aws-single-node-integration-tests
ci/rehearse/openshift/lvm-operator/release-4.21/snyk-deps f8e6129 link unknown /pj-rehearse pull-ci-openshift-lvm-operator-release-4.21-snyk-deps
ci/rehearse/periodic-ci-openshift-lvm-operator-release-4.21-e2e-aws-single-node f8e6129 link unknown /pj-rehearse periodic-ci-openshift-lvm-operator-release-4.21-e2e-aws-single-node
ci/rehearse/periodic-ci-openshift-lvm-operator-release-4.22-e2e-aws-sno-qe-integration-tests 68a2856 link unknown /pj-rehearse periodic-ci-openshift-lvm-operator-release-4.22-e2e-aws-sno-qe-integration-tests
ci/rehearse/periodic-ci-openshift-lvm-operator-release-4.22-e2e-baremetalds-mno-dualstack-qe-integration-tests f8e6129 link unknown /pj-rehearse periodic-ci-openshift-lvm-operator-release-4.22-e2e-baremetalds-mno-dualstack-qe-integration-tests
ci/rehearse/openshift/lvm-operator/release-4.21/e2e-aws f8e6129 link unknown /pj-rehearse pull-ci-openshift-lvm-operator-release-4.21-e2e-aws
ci/rehearse/periodic-ci-openshift-lvm-operator-release-4.21-e2e-baremetalds-sno-dualstack-qe-integration-tests f8e6129 link unknown /pj-rehearse periodic-ci-openshift-lvm-operator-release-4.21-e2e-baremetalds-sno-dualstack-qe-integration-tests
ci/rehearse/openshift/lvm-operator/release-4.22/e2e-aws f8e6129 link unknown /pj-rehearse pull-ci-openshift-lvm-operator-release-4.22-e2e-aws
ci/rehearse/periodic-ci-openshift-lvm-operator-release-4.22-e2e-aws-sno-arm-qe-integration-tests f8e6129 link unknown /pj-rehearse periodic-ci-openshift-lvm-operator-release-4.22-e2e-aws-sno-arm-qe-integration-tests
ci/rehearse/periodic-ci-openshift-lvm-operator-release-4.22-e2e-aws-mno-qe-integration-tests f8e6129 link unknown /pj-rehearse periodic-ci-openshift-lvm-operator-release-4.22-e2e-aws-mno-qe-integration-tests
ci/rehearse/periodic-ci-openshift-lvm-operator-release-4.21-e2e-aws-sno-arm-qe-integration-tests f8e6129 link unknown /pj-rehearse periodic-ci-openshift-lvm-operator-release-4.21-e2e-aws-sno-arm-qe-integration-tests
ci/rehearse/periodic-ci-openshift-lvm-operator-release-4.22-e2e-aws-single-node-integration-tests 68a2856 link unknown /pj-rehearse periodic-ci-openshift-lvm-operator-release-4.22-e2e-aws-single-node-integration-tests
ci/rehearse/periodic-ci-openshift-lvm-operator-release-4.21-e2e-aws-single-node-integration-tests f8e6129 link unknown /pj-rehearse periodic-ci-openshift-lvm-operator-release-4.21-e2e-aws-single-node-integration-tests
ci/rehearse/periodic-ci-openshift-lvm-operator-release-4.21-e2e-aws-mno-arm-qe-integration-tests f8e6129 link unknown /pj-rehearse periodic-ci-openshift-lvm-operator-release-4.21-e2e-aws-mno-arm-qe-integration-tests
ci/rehearse/periodic-ci-openshift-lvm-operator-release-4.22-e2e-aws-mno-arm-qe-integration-tests f8e6129 link unknown /pj-rehearse periodic-ci-openshift-lvm-operator-release-4.22-e2e-aws-mno-arm-qe-integration-tests
ci/rehearse/periodic-ci-openshift-lvm-operator-release-4.21-e2e-baremetalds-mno-dualstack-qe-integration-tests f8e6129 link unknown /pj-rehearse periodic-ci-openshift-lvm-operator-release-4.21-e2e-baremetalds-mno-dualstack-qe-integration-tests
ci/rehearse/periodic-ci-openshift-lvm-operator-release-4.22-e2e-aws f8e6129 link unknown /pj-rehearse periodic-ci-openshift-lvm-operator-release-4.22-e2e-aws
ci/rehearse/periodic-ci-openshift-lvm-operator-release-4.22-e2e-baremetalds-sno-dualstack-qe-integration-tests 68a2856 link unknown /pj-rehearse periodic-ci-openshift-lvm-operator-release-4.22-e2e-baremetalds-sno-dualstack-qe-integration-tests
ci/rehearse/periodic-ci-openshift-lvm-operator-release-4.21-e2e-aws-mno-qe-integration-tests f8e6129 link unknown /pj-rehearse periodic-ci-openshift-lvm-operator-release-4.21-e2e-aws-mno-qe-integration-tests

Full PR test history. Your PR dashboard.

Details

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

approved Indicates a PR has been approved by an approver from all required OWNERS files. jira/valid-reference Indicates that this PR references a valid Jira ticket of any type. lgtm Indicates that a PR is ready to be merged.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants