Skip to content

[shiftstack] Use GitHub for shiftstack-qa clone and fix false-positive test results#3922

Draft
imatza-rh wants to merge 1 commit into
openstack-k8s-operators:mainfrom
shiftstack:shiftstack-qa-github-clone
Draft

[shiftstack] Use GitHub for shiftstack-qa clone and fix false-positive test results#3922
imatza-rh wants to merge 1 commit into
openstack-k8s-operators:mainfrom
shiftstack:shiftstack-qa-github-clone

Conversation

@imatza-rh
Copy link
Copy Markdown
Contributor

@imatza-rh imatza-rh commented May 10, 2026

Summary

Two fixes for the shiftstack CI role:

1. Clone shiftstack-qa from GitHub instead of GerritHub

shiftstack-qa moved code review to GitHub PRs (shiftstack/shiftstack-qa#2). The role default still clones from GerritHub, which is no longer maintained. GerritHub main is 3+ commits behind GitHub main - missing the 4-stable OCP channel switch, lb_tests disable, and .gitreview removal.

  • Change cifmw_shiftstack_qa_repo default to https://github.com/shiftstack/shiftstack-qa
  • Rename cifmw_shiftstack_qa_gerrithub_change to cifmw_shiftstack_qa_change_ref - supports both GitHub PR refs (refs/pull/N/head) and GerritHub change refs (refs/changes/...)

2. Fix false-positive test results in exec_command_in_pod.yml

In tp!2297 build d078ccbd, oc rsh returned rc=0 after 145s while the inner ansible-navigator playbook continued running in the pod for 21 minutes before failing. The shiftstack test reported PASS when OCP was never installed and zero Nova VMs were created. Verified on the live environment (serval70) - inner ansible-log.txt on the pod showed failed=1 at 10:59 UTC, but the outer oc rsh task had already completed with changed: [localhost] at 10:39 UTC.

The fix decouples the test result from the oc exec exit code:

  • Writes the inner command exit code to a marker file on the pod (/tmp/cifmw_cmd_rc)
  • Verifies the actual exit code in a separate oc exec call after the main command returns
  • Polls with retries (up to 2h) to handle cases where the connection drops mid-execution
  • Switches from oc rsh to oc exec for non-interactive pod commands

Test plan

  • GitHub clone validated in tp!2297 build 70b1f455 (5h 8m) - OCP installed (51m), all cluster checks passed, only failed on Octavia LB (known OSPRH-29792)
  • Exit code verification tested on live pod (serval70) - correctly catches false (exit 1) while passing echo hello (exit 0)
  • YAML syntax validated
  • Testproject re-validation with the exit code fix included

@openshift-ci
Copy link
Copy Markdown
Contributor

openshift-ci Bot commented May 10, 2026

Skipping CI for Draft Pull Request.
If you want CI signal for your change, please convert it to an actual PR.
You can still manually trigger a test run with /test all

@imatza-rh imatza-rh force-pushed the shiftstack-qa-github-clone branch 3 times, most recently from 22a3c0d to 4dd8af5 Compare May 11, 2026 13:00
@imatza-rh imatza-rh changed the title [shiftstack] Clone shiftstack-qa from GitHub instead of GerritHub [shiftstack] Use GitHub for shiftstack-qa clone and fix false-positive test results May 11, 2026
@imatza-rh imatza-rh changed the base branch from stable to main May 11, 2026 13:24
…e test results

shiftstack-qa moved code review to GitHub PRs (shiftstack/
shiftstack-qa#2). The role default still clones from
GerritHub, which is no longer maintained.

- Change cifmw_shiftstack_qa_repo default to GitHub
- Rename cifmw_shiftstack_qa_gerrithub_change to
  cifmw_shiftstack_qa_change_ref

Also fix a false-positive bug in exec_command_in_pod.yml
where oc rsh could return rc=0 before the inner command
finished, causing the test to report PASS when the playbook
actually failed. Observed in tp!2297 build d078ccbd - the
shiftstack test ran for only 145s (vs typical 1.5-2h), OCP
was never installed, yet the job reported SUCCESS.

The fix writes the inner command exit code to a marker file
on the pod, then verifies it in a separate oc exec call.
This catches failures even if the first oc exec connection
drops mid-execution. Also switches from oc rsh to oc exec
for non-interactive pod commands.

Related-Issue: #OSPRH-29506

Assisted-By: Claude Code
Signed-off-by: Itay Matza <imatza@redhat.com>
@imatza-rh imatza-rh force-pushed the shiftstack-qa-github-clone branch from 4dd8af5 to b2ab112 Compare May 11, 2026 13:27
@openshift-ci
Copy link
Copy Markdown
Contributor

openshift-ci Bot commented May 11, 2026

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by:
Once this PR has been reviewed and has the lgtm label, please assign imatza-rh for approval. For more information see the Code Review Process.

The full list of commands accepted by this bot can be found here.

Details Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant