Required controlledVocabulary metadata marked as valid while empty

[stevenwinship]

What this PR does / why we need it: Previous PR caused post merge test failures in Jenkins. This PR includes the code from the previous PR plus the fixes needed to pass the tests

Which issue(s) this PR closes:#11900

• Closes Required controlledVocabulary metadata marked as valid while empty #11900

Special notes for your reviewer: Original PR #11950

Suggestions on how to test this:

Does this PR introduce a user interface change? If mockups are available, please link/include them here:

Is there a release notes update needed for this change?:

Additional documentation:

[coveralls]

coverage: 24.95% (-0.001%) from 24.951% — 11900-improved-cvoc-value-validation2 into develop

[cmbz]

2026-04-22: @ErykKul we're returning this PR back to you for further fixes.

[ErykKul]

Root cause: allowHarvestingMissingCVV was gating the sanitizer, but it should only affect the parser.

The parser uses the flag to decide whether to throw on an unmappable controlled vocabulary string. The sanitizer handles a different case: required controlled vocabulary fields that come up empty after initDatasetFields() because the source format (e.g. oai_dc) does not carry them. Those fields must always be filled with a proper ControlledVocabularyValue wrapping "N/A", not a plain "N/A" string, otherwise the second validation pass still fails.

The fix removes the gate and routes by field type instead. Controlled vocabulary fields get a ControlledVocabularyValue("N/A"), other fields get setSingleValue("N/A"). With that, allowHarvestingMissingCVV=false correctly imports 7 of 8 records: the parser rejects the 1 record with a bad controlled vocabulary string, and the sanitizer rescues the 7 records with empty controlled vocabulary fields.

[rtreacy]

Looks good generally. Passes tests. On coding style, is it really necessary/desirable to use import edu.harvard.iq.dataverse.*; ?

[stevenwinship]

QAing this pr I found the following issue:

Setup: JVM setting -Ddataverse.api.allow-incomplete-metadata=true (same result if not set)
Run HarvestingClientsIT

Create a new dataset in UI and observe the "Subject" drop down. All the 'N/A' values that were assigned in the harvesting (ImportServiceBean sanitize) were also added to the drop down list.

[jp-tosca]

Let us know when you can look at this @ErykKul :) thank you!

[ErykKul]

The previous fix created new ControlledVocabularyValue(null, "N/A", f.getDatasetFieldType()), a transient JPA entity pointing to the actual field type (e.g. Subject). Because DatasetField.controlledVocabularyValues has @ManyToMany(cascade = {CascadeType.MERGE}), merging the DatasetField on save cascaded to this new entity and inserted it as a permanent row in controlledvocabularyvalue with the Subject's datasetfieldtype_id. After running HarvestingClientsIT, every CV field that was sanitized with N/A added a new "N/A" entry to its dropdown. Running the test repeatedly multiplied the rows.

The fix uses datasetfieldService.findNAControlledVocabularyValue() instead, which returns the existing managed singleton (the same entity SWORD uses) that has datasetFieldType = null. Merging a managed entity only updates the join table — no new row is ever inserted into controlledvocabularyvalue, and the Subject dropdown stays clean. The if (naValue != null) guard handles the rare case where the singleton was never loaded via loadNAControlledVocabularyValue; in that case the field is left empty and the second-pass validation will throw ImportException, which is correct. If the N/A placeholder doesn't exist, the record cannot be sanitized and should be skipped.

[github-actions]

📦 Pushed preview images as

ghcr.io/gdcc/dataverse:11900-improved-cvoc-value-validation2

ghcr.io/gdcc/configbaker:11900-improved-cvoc-value-validation2

🚢 See on GHCR. Use by referencing with full name as printed above, mind the registry name.