Skip to content

Fix tasmax all-1e20 path in monthly formula workflow#459

Merged
rbeucher merged 5 commits into
mainfrom
fix/tasmax-1e20-alignment
Jun 24, 2026
Merged

Fix tasmax all-1e20 path in monthly formula workflow#459
rbeucher merged 5 commits into
mainfrom
fix/tasmax-1e20-alignment

Conversation

@rbeucher

Copy link
Copy Markdown
Member

This pull request improves the handling of time coordinate alignment and missing value markers in the data processing pipeline, ensuring more robust and accurate calculations when working with temporal data and reductions. The main changes add logic to detect and handle shifted time labels, introduce masking for missing value sentinels during reductions, and expand test coverage for these scenarios.

Time coordinate alignment improvements:

  • Enhanced the select_and_process_variables method in atmosphere.py to detect when time labels have changed (even if the length is unchanged), ensuring that datasets are rebuilt rather than misaligned, which prevents data from being replaced with NaNs when time coordinates shift.
  • Added a unit test (test_formula_same_time_length_but_shifted_labels_rebuilds) to verify that shifted time labels trigger a proper dataset rebuild and maintain data integrity.

Handling of missing value sentinels in reductions:

  • Introduced the _mask_missing_values_for_reduction helper in calc_utils.py to mask out missing value markers (like 1e20 or NaN) before performing reductions, ensuring that these sentinels do not affect the results of monthly minimum and maximum calculations.
  • Updated both calculate_monthly_minimum and calculate_monthly_maximum functions to apply this masking step before reduction, preventing missing value markers from dominating reduction results [1] [2].
  • Added a unit test (test_ignores_fill_value_marker_in_maximum) to confirm that fill value markers are properly ignored in monthly maximum calculations.

@codecov

codecov Bot commented Jun 24, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 93.02326% with 3 lines in your changes missing coverage. Please review.
✅ Project coverage is 76.9%. Comparing base (9316f47) to head (95c8d8a).
⚠️ Report is 1 commits behind head on main.

Files with missing lines Patch % Lines
src/access_moppy/atmosphere.py 40.0% 3 Missing ⚠️
Additional details and impacted files
@@           Coverage Diff           @@
##            main    #459     +/-   ##
=======================================
+ Coverage   76.7%   76.9%   +0.2%     
=======================================
  Files         31      31             
  Lines       5927    5974     +47     
  Branches    1094    1107     +13     
=======================================
+ Hits        4546    4594     +48     
  Misses      1131    1131             
+ Partials     250     249      -1     
Flag Coverage Δ
unit 76.9% <93.0%> (+0.2%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@rbeucher rbeucher merged commit 7142b03 into main Jun 24, 2026
4 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant