Skip to content

[oadp-1.4] Fix IBU delay on SNO by waiting for CRDs and disabling leader election #2086

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Open
shubham-pampattiwar wants to merge 1 commit into
base: oadp-1.4
Choose a base branch
from
Open

[oadp-1.4] Fix IBU delay on SNO by waiting for CRDs and disabling leader election #2086

shubham-pampattiwar wants to merge 1 commit into from
+626 −11

Conversation

@shubham-pampattiwar
Copy link
Member

@shubham-pampattiwar shubham-pampattiwar commented Feb 10, 2026 •
edited
Loading

Summary

Cherry-pick of #2082 to oadp-1.4 branch.

Fixes ~8 minute delay during Image-Based Upgrade (IBU) on SNO clusters by:

  • Waiting for Route/SCC CRDs before starting controller
  • Disabling leader election on SNO (single replica, no benefit)
  • Adding LeaderElectionReleaseOnCancel for faster recovery on crash

See https://issues.redhat.com/browse/OADP-7419

Test plan

  • Unit tests pass
  • IBU testing on SNO cluster

🤖 Generated with Claude Code

@shubham-pampattiwar @claude
Fix IBU delay on SNO by waiting for CRDs and disabling leader election (
7a77aa0 
openshift#2082)

During Image-Based Upgrade (IBU) on Single Node OpenShift (SNO) clusters,
the OADP controller was experiencing an ~8 minute delay before reaching
DPA Reconciled=True. This was caused by:

1. Controller crashing when Route/SCC CRDs weren't available during
   cluster initialization (2 min cache sync timeout)
2. New instance waiting for leader lease to expire (4.5 min on SNO)

This commit implements three complementary fixes:

1. CRD availability wait: Before starting the controller, poll the
   discovery API for Route and SCC CRDs. This prevents the crash
   by waiting until external OpenShift operators have registered
   their CRDs.

2. LeaderElectionReleaseOnCancel: Release the leader lease when the
   controller crashes, allowing immediate restart without waiting
   for lease expiry.

3. Disable leader election on SNO: Since SNO has only one node and
   the operator runs with replicas=1, leader election provides no
   benefit and only adds overhead.

Combined effect: SNO IBU delay reduced from ~8 min to < 1 min.

Fixes: https://issues.redhat.com/browse/OADP-7419

Co-authored-by: Claude <noreply@anthropic.com>
@coderabbitai
Copy link
Contributor

coderabbitai bot commented Feb 10, 2026 •
edited
Loading

Important

Review skipped

Auto reviews are limited based on label configuration.

🚫 Review skipped — only excluded labels are configured. (1)
  • do-not-merge/work-in-progress

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

  • 🔍 Trigger review
✨ Finishing touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Post copyable unit tests in a comment

Comment @coderabbitai help to get the list of available commands and usage tips.

@openshift-ci openshift-ci bot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Feb 10, 2026
@openshift-ci
Copy link

openshift-ci bot commented Feb 10, 2026

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: kaovilai, shubham-pampattiwar

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Details Needs approval from an approver in each of these files:
  • OWNERS [kaovilai,shubham-pampattiwar]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@openshift-ci
Copy link

openshift-ci bot commented Feb 10, 2026

@shubham-pampattiwar: all tests passed!

Full PR test history. Your PR dashboard.

Details

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here.

@weshayutin
Copy link
Contributor

weshayutin commented Feb 10, 2026

@shubham-pampattiwar noting oadp-1.4 is currently under test for release. If the current QE build fails and this merges we MAY need to revert...
I suggest we hold for a week.
/HOLD

@openshift-ci openshift-ci bot added the do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. label Feb 10, 2026
@shubham-pampattiwar
Copy link
Member Author

shubham-pampattiwar commented Feb 10, 2026

@weshayutin sounds good !

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@kaovilai kaovilai kaovilai approved these changes

@eemcmullan eemcmullan Awaiting requested review from eemcmullan

@savitharaghunathan savitharaghunathan Awaiting requested review from savitharaghunathan

At least 2 approving reviews are required to merge this pull request.

Assignees

No one assigned

Labels

approved Indicates a PR has been approved by an approver from all required OWNERS files. do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command.

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

3 participants

@shubham-pampattiwar @weshayutin @kaovilai