Fix Oyr ocean resampling pipeline (enable resampling, yearly time/bounds/filename, calendar)#463
Fix Oyr ocean resampling pipeline (enable resampling, yearly time/bounds/filename, calendar)#463rhaegar325 wants to merge 4 commits into
Conversation
Codecov Report❌ Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## main #463 +/- ##
=======================================
+ Coverage 76.9% 77.2% +0.3%
=======================================
Files 31 31
Lines 5974 6024 +50
Branches 1107 1119 +12
=======================================
+ Hits 4594 4648 +54
+ Misses 1131 1126 -5
- Partials 249 250 +1
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
|
Can you link the issues that this will close? |
|
How much of that also applied to ESM1-6 and CMIP7. I think we should check to make sure that we don't just use code for CMIP6. |
|
Most of this PR should apply to ACCESS-ESM1-6 and CMIP7, since the resampling/time-bounds/calendar fixes are in shared code paths used after CMIP7→CMIP6 mapping. Two things to double-check: CMIP7 yearly filename formatting (this change looks CMIP6-only), and OM2 source_id acceptance for ACCESS-ESM1-6 so ocean CMORisation isn’t blocked independently. |
PR Summary: Fix Oyr ocean resampling pipeline (enable resampling, yearly time/bounds/filename, calendar)
Overview
This change set fixes a chain of defects that prevented ACCESS-ESM1-5 ocean yearly
(
Oyr) variables from being CMORised correctly, and tightens CF/WCRP compliance of theresampled output. All fixes are in the producer (
access_moppy); each is covered by unittests and verified end-to-end against real model output with the WCRP CMIP6
compliance-checker.
Problems fixed
Oyr/Omonfiles have a danglingtime:boundsbut notime_bndsvariable (compliance ATTR004/VAR004)base.py,atmosphere.py,ocean.py(landed earlier, #455)Oyroutput still had 12 monthly steps instead of yearlyenable_resamplingto the ocean CMORisers, andOcean_CMORiser_OM2/OM3.__init__did not accept itdriver.py,ocean.pycalculate_time_boundscould not infer frequency from one pointutilities.py,base.pyresamplelabels bins with theYE(year-end) boundaryutilities.pytime:units/calendar(CF-invalid)decode_cfmoved units into encoding and they were never restoredutilities.py010107-...)YYYYMMbranchvocabulary_processors.pyglob) file inputs crashed: "Index must be monotonic for resampling"utilities.py[TIME001]yearly midpoint off by 1 day;time_bndsvstimecalendar mismatch"GREGORIAN"label was read by cftime as the Julian/Gregorianstandardcalendar, while the file declaresproleptic_gregorianutilities.pyChanges
src/access_moppy/driver.pyACCESS_ESM_CMORiser.enable_resamplingdefaultFalse→True(no-op unless there isa genuine frequency mismatch).
validate_frequency/enable_resampling/resampling_methodto bothOcean_CMORiser_OM2andOcean_CMORiser_OM3.src/access_moppy/ocean.pyOcean_CMORiser_OM2/OM3.__init__accept and forward the three resampling parameters tothe base class (the link was previously broken at the subclass level).
src/access_moppy/utilities.py_shift_resampled_time_to_period_midpoint: centre the resampled time coordinate on theperiod midpoint (CMOR convention), reusing the existing per-period bounds helpers.
resample_dataset_temporal: sort by time before resampling (monotonicity); normalise the"GREGORIAN"calendar label to"proleptic_gregorian"before decoding; restore the CFunits/calendarencoding (numeric) after resampling._normalise_calendar_name: map only the non-CF"GREGORIAN"to"proleptic_gregorian".calculate_time_bounds: add afreq_hintfallback for single-point axes; normalise thecalendar before date arithmetic.
src/access_moppy/base.py_target_frequency_hint: derive adaily/monthly/yearlyhint from the CMOR table.calculate_time_boundsfor the time axis.src/access_moppy/vocabulary_processors.pyOyrtime range is year-only (YYYY-YYYY).Behaviour change
enable_resamplingnow defaults toTrue. Resampling is a no-op when the inputfrequency already matches the target (e.g. monthly→monthly), and only triggers on a real
mismatch (e.g. monthly input for an
Oyrtable). Passenable_resampling=Falseto optout. Sea-ice is unaffected (the driver does not wire the flag into it); no existing test
depended on the old default.
Tests
test_utilities.py:TestResampleTimeMidpoint: midpoint on ~2 July (incl. leap year), CF units preserved,unsorted input handled.
test_single_point_with_freq_hint,test_gregorian_label_treated_as_proleptic,test_normalise_calendar_name.test_vocabulary_processors.py:test_generate_filename_yearly_year_only.test_utilities/test_ocean/test_atmosphere/test_driver/test_vocabulary_processors(the 7 failingtest_cmip7_*cases are apre-existing CMIP7 CV submodule issue, unrelated to this change).
module use /g/data/xp65/public/modules && module load conda/analysis3 PYTHONPATH=src python -m pytest tests/unit/test_utilities.py tests/unit/test_ocean.py \ tests/unit/test_driver.py tests/unit/test_vocabulary_processors.py -qCompliance verification
Run against a real CMORised file
(
no3_Oyr_ACCESS-ESM1-5_historical_r1i1p1f1_gn_0101-0110.nc):time_bndspresent (ATTR004/VAR004 cleared).[TIME001]time-squareness passes —time[0] = 36706.5, the proleptic_gregorianyearly midpoint the checker expects.
time_bnds/timecalendar mismatch cleared (bothproleptic_gregorian).YYYY-YYYY.