Skip to content

KVM NAS backup: resume VM and exit on backup failure#12872

Open
jmsperu wants to merge 1 commit intoapache:4.22from
jmsperu:fix/nasbackup-resume-vm-on-failure
Open

KVM NAS backup: resume VM and exit on backup failure#12872
jmsperu wants to merge 1 commit intoapache:4.22from
jmsperu:fix/nasbackup-resume-vm-on-failure

Conversation

@jmsperu
Copy link

@jmsperu jmsperu commented Mar 20, 2026

Summary

Fix three bugs in nasbackup.sh that caused VMs to remain paused indefinitely when backup jobs fail (e.g. storage full, I/O error):

  1. Infinite polling loop: backup_running_vm() falls through the Failed case without exiting, causing the script to poll the already-failed job forever. Fixed by adding exit 1 after cleanup.

  2. Continued processing after failure: backup_stopped_vm() continues processing subsequent disks after qemu-img convert fails. Fixed by adding exit 1 after cleanup.

  3. VM never resumed: cleanup() removes temp files and unmounts but never resumes the VM that was paused by virsh backup-begin. Fixed by adding a VM state check and virsh resume at the top of cleanup().

Root Cause

When virsh backup-begin starts a backup, the VM may be paused. If the backup fails for any reason (storage full, network issue, I/O error), the script's cleanup path never calls virsh resume, leaving the VM paused until manual intervention.

Test Plan

  • Trigger a backup with insufficient NAS storage — verify VM is resumed after failure
  • Trigger a backup with NAS unmounted mid-backup — verify VM is resumed
  • Normal backup completes successfully (no regression)
  • Verify qemu-img convert failure on stopped VM exits cleanly

Fixes #12821

Fix three bugs in nasbackup.sh that caused VMs to remain paused
indefinitely when backup jobs fail (e.g. storage full):

1. Add exit after cleanup on Failed backup job status to prevent
   infinite polling loop in backup_running_vm()
2. Add exit after cleanup on qemu-img convert failure in
   backup_stopped_vm() to stop processing subsequent disks
3. Add VM state check and virsh resume to cleanup() so paused VMs
   are automatically resumed after backup failure

Fixes apache#12821
@codecov
Copy link

codecov bot commented Mar 22, 2026

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 3.70%. Comparing base (e93ae1a) to head (8550e5a).

❗ There is a different number of reports uploaded between BASE (e93ae1a) and HEAD (8550e5a). Click for more details.

HEAD has 1 upload less than BASE
Flag BASE (e93ae1a) HEAD (8550e5a)
unittests 1 0
Additional details and impacted files
@@              Coverage Diff              @@
##               4.22   #12872       +/-   ##
=============================================
- Coverage     17.61%    3.70%   -13.91%     
=============================================
  Files          5917      448     -5469     
  Lines        531430    38028   -493402     
  Branches      64973     7036    -57937     
=============================================
- Hits          93586     1409    -92177     
+ Misses       427288    36432   -390856     
+ Partials      10556      187    -10369     
Flag Coverage Δ
uitests 3.70% <ø> (ø)
unittests ?

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant