iris.std_names.STD_NAMES check does not include CMIP7 standard names

[k-a-webb]

The step within cmor/fix routine which ensures that a dataset's standard name is in the iris.std_names.STD_NAMES can fail for some CMIP7 variables, presumably because the library has not been updated in the current version of iris used by ESMValCore/Tool v2.14.0 (i.e the latest, v3.15.0).

For example, the CMIP7 entry for od550csaer_tavg-u-hxy-u, https://github.com/ESMValGroup/ESMValCore/blob/main/esmvalcore/cmor/tables/cmip7/tables/CMIP7_aerosol.json#L2141

  File "/fs/site8/eccc/crd/cccma/users/rkw001/code/esmvalcore/esmvalcore_v2.14_a4d/esmvalcore/preprocessor/__init__.py", line 563, in _run_preproc_function
    return function(items, **kwargs)
  File "/fs/site8/eccc/crd/cccma/users/rkw001/code/esmvalcore/esmvalcore_v2.14_a4d/esmvalcore/cmor/fix.py", line 204, in fix_metadata
    cube_list = fix.fix_metadata(cube_list)
  File "/fs/site8/eccc/crd/cccma/users/rkw001/code/esmvalcore/esmvalcore_v2.14_a4d/esmvalcore/cmor/_fixes/fix.py", line 395, in fix_metadata
    cube = self._fix_standard_name(cube)
  File "/fs/site8/eccc/crd/cccma/users/rkw001/code/esmvalcore/esmvalcore_v2.14_a4d/esmvalcore/cmor/_fixes/fix.py", line 509, in _fix_standard_name
    cube.standard_name = self.vardef.standard_name
    ^^^^^^^^^^^^^^^^^^
  File "/fs/site8/eccc/crd/cccma/users/rkw001/envs/a4d_env_esmvalv214/lib/python3.13/site-packages/iris/common/mixin.py", line 259, in standard_name
    self._metadata_manager.standard_name = _get_valid_standard_name(name)
                                           ~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^
  File "/fs/site8/eccc/crd/cccma/users/rkw001/envs/a4d_env_esmvalv214/lib/python3.13/site-packages/iris/common/mixin.py", line 56, in _get_valid_standard_name
    raise ValueError("{!r} is not a valid standard_name".format(name))
ValueError: 'atmosphere_optical_thickness_due_to_ambient_aerosol_particles_assuming_clear_sky' is not a valid standard_name

This looks to be an iris issue,

import iris.std_names

In [3]: "atmosphere_optical_thickness_due_to_ambient_aerosol_particles_assuming_clear_sky" in iris.std_names.STD_NAMES
Out[3]: False

In [7]: iris.__version__
Out[7]: '3.15.0'

However, relevant to ESMValCore is that I cannot get around this check even when setting cmor_table/strict: false

[k-a-webb]

For this case, there is an open CF issue (cf-convention/vocabularies#258) where the standard name seems to be approved, but the issue was not closed. Likely a case of pending acceptance.

[bouweandela]

Indeed, it looks like this is not a valid standard name (yet). You may want to comment on that issue to alert those involved.

As a workaround to get your data loaded with Iris, you could implement a fix that sets the standard name to None.

[bouweandela]

I compared the standard names from the CMIP7 CMOR tables with those in the CF conventions and reported discrepancies in CMIP-Data-Request/CMIP7_DReq_Content#23.

Details

Code to get the list of standard names from the CMIP7 CMOR tables:

{entry["standard_name"] for path in Path("tables").glob("*.json") for entry in json.loads(path.read_text()).get("variable_entry", {}).values()}

Code to get the list of standard names from the CF conventions:

wget https://cfconventions.org/Data/cf-standard-names/current/src/cf-standard-name-table.xml

{entry.attrib["id"] for key in ("entry", "alias") for entry in ET.parse("cf-standard-name-table.xml").getroot().iterfind(key)}

[k-a-webb]

Thanks!
I was able to get the recipe to run by setting check_level: relaxed. I agree that your suggestion of writing a fix would be a better solution, however.