Add CMIP7 QC — per-variable rules for all 293 ESM1-6 variables + batch report integration

[rbeucher]

What's this?

QC (quality control) for CMIP7 CMORised output. The idea is simple: after we write a NetCDF file, we want to sanity-check that the data is physically plausible — right units, no junk values, temperature actually looks like temperature, etc.

Why?

We had some basic infra in place but it only really covered tas with hardcoded experiment limits. This PR generalises that to all 293 ACCESS-ESM1-6 mapped variables and plugs QC results directly into the batch report so it's not a separate thing you have to remember to run.

How does it work?

Per-variable rules (cmip7_ranges.yml)

Every variable in the ESM1-6 mapping now has an explicit entry with:

• its expected units
• a default physical min/max (derived from the variable's unit type and positive direction)
• experiment-specific overrides (historical / piControl / ssp*)

The ranges weren't invented — they come from the mapping definitions themselves. E.g. evspsblsoi has units kg m-2 s-1 and positive: up, so its minimum is clamped to 0 (upward flux can't be negative). tas keeps its custom hand-tuned limits. Everything else is auto-derived.

Batch report integration

When moppy-batch-report runs, it now scans the output folder for .nc files and adds a qc block to the JSON report:

{
  "qc": {
    "passed": 42,
    "failed": 2,
    "total": 44,
    "failures": [
      {
        "file": "/output/tas.nc",
        "variable_id": "tas",
        "experiment_id": "piControl",
        "observed_range": [182.0, 329.4],
        "allowed_range": [180.0, 325.0],
        "units": "K"
      }
    ]
  }
}

If you don't want it, set MOPPY_SKIP_QC=1 or pass --skip-qc to the CLI.

New moppy-qc CLI (added in earlier commit on this branch)

moppy-qc /path/to/output/*.nc
# exits 0 if all pass, 1 if any fail

What's tested?

11 unit tests covering:

• pass/fail range checks for individual variables
• experiment-specific rule matching (piControl limits tighter than historical)
• positive-direction enforcement (negative upward flux → fail)
• units mismatch detection
• CLI exit codes
• coverage guard ensuring all 293 mapped variables have explicit YAML entries
• CMORiser write-path integration

Files changed

• src/access_moppy/qc/cmip7.py — main validator, validate_cmip7_output_detailed() for non-raising batch use
• src/access_moppy/resources/qc/cmip7_ranges.yml — 293 per-variable rule blocks
• src/access_moppy/batch_report.py — QC section added to batch report
• tests/unit/test_cmip7_qc.py — full unit test coverage
• docs/source/qc_validation.rst — user-facing docs (notebook API, CLI, batch report, how to extend)

[codecov]

Codecov Report

❌ Patch coverage is 81.73077% with 57 lines in your changes missing coverage. Please review.
✅ Project coverage is 77.3%. Comparing base (7142b03) to head (f2d1a2c).

Files with missing lines	Patch %	Lines
src/access_moppy/qc/cmip7.py	79.8%	31 Missing and 17 partials ⚠️
src/access_moppy/batch_report.py	84.8%	2 Missing and 3 partials ⚠️
src/access_moppy/utilities.py	91.7%	1 Missing and 1 partial ⚠️
src/access_moppy/vocabulary_processors.py	83.3%	2 Missing ⚠️

❌ Your patch check has failed because the patch coverage (81.7%) is below the target coverage (90.0%). You can increase the patch coverage or adjust the target coverage.

Additional details and impacted files

@@           Coverage Diff           @@
##            main    #456     +/-   ##
=======================================
+ Coverage   76.9%   77.3%   +0.4%     
=======================================
  Files         31      33      +2     
  Lines       5974    6277    +303     
  Branches    1107    1169     +62     
=======================================
+ Hits        4594    4850    +256     
- Misses      1131    1154     +23     
- Partials     249     273     +24     

Flag	Coverage Δ
unit	77.3% <81.7%> (+0.4%)	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.