Document the non-contiguous `IIASA-IAMC` ScenarioMIP datasets

[penguian]

Dataset source ID(s)

IIASA-IAMC-h-1-1-1 (and other future IIASA-IAMC ScenarioMIP source IDs)

Describe the issue

In the future ScenarioMIP anthropogenic emissions datasets (e.g., SO2_em_anthro), the NetCDF metadata designates the frequency as "mon" (monthly).

However, because these projections are produced using Integrated Assessment Models (IAMs) that solve at 5-yearly intervals, the files only contain monthly data for the reporting milestones (e.g., monthly data for 2022–2025, followed by 2030, 2035, ..., 2100). The intervening years are omitted.

For a file spanning the range 202201-210012.nc:

• A continuous monthly dataset would contain 948 months.
• The raw IIASA-IAMC files contain only 228 months.

Standard processing tools and libraries (such as the UK Met Office's ants or mule packages) that expect continuous time boundaries will fail when trying to ingest these files, raising errors such as:
ValueError: F03 ancillaries with time bounds are required to be contiguous

Expected data

Clarification in the official input4MIPs documentation explaining that:

1. ScenarioMIP future projection datasets report seasonal/monthly data at 5-yearly intervals rather than as a continuous monthly time series.
2. Modeling centers and users will need to perform temporal interpolation to reconstruct a continuous monthly series if their downstream pipelines require contiguous time boundaries.

Screenshots

N/A

Additional context

For example, the input4MIPs readthedocs overview page for anthropogenic aerosol emissions focuses heavily on the historical CEDS dataset (which is continuous).

Without documentation clarifying the 5-yearly structure of the future IAM scenarios, new users have no reason to expect these gaps and only discover them when downstream pipelines crash.

[znichollscr]

Hi @penguian, thanks for the report. We can definitely add to the docs (also feel free to make the edits you would like to see to https://github.com/PCMDI/input4MIPs_CVs/blob/main/docs/dataset-overviews/anthropogenic-slcf-co2-emissions.md and https://github.com/PCMDI/input4MIPs_CVs/blob/main/docs/dataset-overviews/open-biomass-burning-emissions.md and make a pull request if you want things to move faster).

In the future ScenarioMIP anthropogenic emissions datasets (e.g., SO2_em_anthro), the NetCDF metadata designates the frequency as "mon" (monthly).

Do you have any docs which indicate that this labelling is incorrect? My understanding (maybe @taylor13 can weigh in here too) is that the frequency label is about the size of the timesteps. It is correct to say that this data is sampled monthly: each timestep is one month long. I have never seen anything which says, "You have to cover every month between your start and end times". I can fully appreciate how you could assume that this would be the case, but what I'm trying to get at is whether that is an assumption, or whether there is some spec out there that we aren't aware of that we should be following, but aren't (if that's the case, then we will also add that guidance to our docs to say, "We didn't know about this spec, be careful with the 'frequency' attribute, it is inconsistent with the spec."). It has been like this since CMIP6 (where these scenario emissions were also reported only for every 5th year), so if we've been doing this incorrectly, we've been doing it incorrectly for a very long time.

Without documentation clarifying the 5-yearly structure of the future IAM scenarios, new users have no reason to expect these gaps and only discover them when downstream pipelines crash

Unfortunately I have to disagree with you here. I don't think we have ever set out the expectation of annual timesteps, and I don't know of any documentation that makes this clear either (see the point above). If you can point me to such documents or standards, I will gladly be corrected and we will add clarifications to users in our docs that they have to check this themselves first, rather than assuming there are no gaps.

[vnaik60]

I agree with @znicholls. There is no expectation that the scenario emissions have to be in annual timesteps. In CMIP5, they were in 10 year intervals and in CMIP6 and CMIP7 they are in 5 year intervals. At GFDL, we do not need to interpolate to create continuous monthly timeseries files since our in-built time interpolator in the model takes care of this.

[penguian]

Thanks for your replies. I agree that "mon" does not imply contiguity in time.

As a research software engineer, I usually work directly from the documentation for the current project (in this case CMIP7) plus anything relevant in historical context (in this case CMIP6).

I did not see anything about the non-contiguity in time until I discovered it while running Python scripts that use Iris.

I asked Google AI mode where I should look for documentation that explains the non-contiguity and it produced the following summary:

Technical documentation for input4MIPs does not explicitly define the non-contiguous, 5-yearly structure of future scenario datasets, using the frequency="mon" attribute to represent the outer temporal domain rather than continuous data points ... The native 5-year resolution and required interpolation are instead specified in the IIASA ECE IAMC framework, the CMIP6 forcing methodology by Feng et al. (2021), and individual ESM model documentation. [1]

[1] [https://pmc.ncbi.nlm.nih.gov](https://pmc.ncbi.nlm.nih.gov/articles/PMC8917170/)

I then examined this reference. It includes the statement:

Feng et al.20 provided spatially distributed anthropogenic emissions historical data with annual resolution and future scenario data in 10-year intervals for CMIP6.

Feng et al. (2021) itself says:

The second major difference is that future emissions are not produced annually but are provided for 2015, 2020, and at decadal intervals thereafter. This is because long-term models do not provide annual data. The choice was made to only distribute data for years that were originally produced by the models and not interpolated data.

So if I had read this CMIP6 paper to begin with, or if I had been directly involved in processing of CMIP6 forcing datasets, I would have expected at least some CMIP7 scenario datasets to be non-contiguous in time.

[penguian]

To summarize, I don't see this as a dataset issue but as a documentation issue for developers who are unfamiliar with the CMIP6 scenario datasets and documentation.

[penguian]

Rather than my directly editing the Markdown files, I would just suggest some wording along the lines of:

Please note that some of the IIASA-IAMC datasets for ScenarioMIP are not contiguous in time, but instead provide monthly data for specific years in the period 2022-2100.

That would be sufficient.

[taylor13]

(Responding to #447 (comment)): It sounds like all now agree (and I certainly do) that the "frequency" records the reporting interval between successive time slices, but the periods over which data are reported do not need to be contiguous.

(Note that in the CMIP7 data request, there are several variables characterizing the monthly mean diurnal cycles. For these variables, the frequency is 1 hr (diurnal cycle resolved hourly), but between each group of 24 time-slices there is a break of 1 month, so there are 24x12 time slices reported each year, but they are not evenly spaced.)

[znichollscr]

@penguian thanks for the clear feedback, super helpful. Please take a look at #448 and comment there with any changes or confirmation that this is the kind of guidance you were thinking of.

@taylor13 a couple of things to check

"frequency" records the reporting interval between successive time slices

Update to '"frequency" records the reporting interval between successive (contiguous) time slices' ? Just to be clear that we don't consider non-contiguous jumps in this calculation?

Could it be phrased more simply as '"frequency" records the length of each timestep (i.e. the difference between the start and end of each time bound pair)'? Or am I missing some use case?