Make restart on ES_MDA work with changed ensamble size

[erlenlh]

Issue
Resolves #12180

Approach
Created a function for re-mapping lists into sets, finding common indices and converting back to list for further processing. Comparing sets eliminates the problem of different length lists.

(Screenshot of new behavior in GUI if applicable)

• PR title captures the intent of the changes, and is fitting for release notes.
• Added appropriate release note label
• Commit history is consistent and clean, in line with the contribution guidelines.
• Make sure unit tests pass locally after every commit (git rebase -i main --exec 'just rapid-tests')

When applicable

• When there are user facing changes: Updated documentation
• New behavior or changes to existing untested code: Ensured that unit tests are added (See Ground Rules).
• Large PR: Prepare changes in small commits for more convenient review
• Bug fix: Add regression test for the bug
• Bug fix: Add backport label to latest release (format: 'backport release-branch-name')

[codecov-commenter]

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 90.35%. Comparing base (3b015c8) to head (327bb52).
⚠️ Report is 1 commits behind head on main.
✅ All tests successful. No failed tests found.

Additional details and impacted files

@@            Coverage Diff             @@
##             main   #12909      +/-   ##
==========================================
- Coverage   90.36%   90.35%   -0.01%     
==========================================
  Files         453      453              
  Lines       31153    31167      +14     
==========================================
+ Hits        28150    28161      +11     
- Misses       3003     3006       +3     

Flag	Coverage Δ
cli-tests	37.23% <75.00%> (+0.01%)	⬆️
gui-tests	67.81% <68.75%> (-0.01%)	⬇️
performance-and-unit-tests	77.03% <100.00%> (+<0.01%)	⬆️
test	46.16% <6.25%> (-0.02%)	⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[codspeed-hq]

Merging this PR will not alter performance

✅ 35 untouched benchmarks

---

_{Comparing Fix_restart_ESMDA_when_size_change (327bb52) with main (3b015c8)}

[jonathan-eq]

Does this solve the issue? I think we need a larger gui test where we run ert with design_matrix realization 0-9, close ert and remove realization 8 and 9 from design matrix, open ert and try "Restart run" from the previous run.
I thought the issue was that the validator uses the ensemble_size from config/design_matrix on "restart from", instead of what was actually used in the run that we are restarting from 🤔

[jonathan-eq]

There should already be an integration test that you can backpack off, otherwise I can help you get started :)

[xjules]

Does this solve the issue? I think we need a larger gui test where we run ert with design_matrix realization 0-9, close ert and remove realization 8 and 9 from design matrix, open ert and try "Restart run" from the previous run. I thought the issue was that the validator uses the ensemble_size from config/design_matrix on "restart from", instead of what was actually used in the run that we are restarting from 🤔

I'd say in this case, it should be enough to test it manually. The complexity of such a test (ie. restart esmda) would be much higher than the frequency of the actual cases that could have this issue.

[jonathan-eq]

Does this solve the issue? I think we need a larger gui test where we run ert with design_matrix realization 0-9, close ert and remove realization 8 and 9 from design matrix, open ert and try "Restart run" from the previous run. I thought the issue was that the validator uses the ensemble_size from config/design_matrix on "restart from", instead of what was actually used in the run that we are restarting from 🤔

I'd say in this case, it should be enough to test it manually. The complexity of such a test (ie. restart esmda) would be much higher than the frequency of the actual cases that could have this issue.

I am sure we have other tests that just create some mock storage of some kind. We only need to test this one panel or even just the validator.

[erlenlh]

Does this solve the issue? I think we need a larger gui test where we run ert with design_matrix realization 0-9, close ert and remove realization 8 and 9 from design matrix, open ert and try "Restart run" from the previous run. I thought the issue was that the validator uses the ensemble_size from config/design_matrix on "restart from", instead of what was actually used in the run that we are restarting from 🤔

How I read the issue, its not about realizations from design matrix. Its about changing num_realizations in config to a lower value. Since ens_mask is loaded from prior_storage is has the old length. Could also be issues if we tamper with design matrix, I havent considered that since I didnt interpret the issue that way 🤷

[Copilot]

Pull request overview

This PR fixes a bug where restarting an ES_MDA run would fail when the ensemble size in the configuration was changed to a different value than the original run. The issue occurred because the code attempted to use bitwise AND (&=) on boolean arrays of different lengths, which is not supported.

Changes:

• Introduced _create_combined_ensemble_mask function that handles mask intersection using set operations on indices instead of array operations, allowing masks of different lengths to be properly combined
• Updated smoother_update to use the new function instead of in-place bitwise AND
• Added comprehensive unit tests covering various scenarios including different-length masks

Reviewed changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated 2 comments.

File	Description
src/ert/analysis/_es_update.py	Added _create_combined_ensemble_mask function to safely intersect ensemble masks of different lengths; refactored smoother_update to use the new function
tests/ert/unit_tests/analysis/test_es_update.py	Added import for the new function and parametrized unit tests covering None case, same-length intersection, different-length intersection, and no-intersection scenarios

[frode-aarstad]

Looks good. Only comment is the testcase suggested by copilot. The failing codspeed test is probably just flaky

[xjules]

I think it should be ready now. 🚀