Skip to content

NaN-handling for Energy Score#123

Open
frazane wants to merge 5 commits into
mainfrom
energy-nan-handling
Open

NaN-handling for Energy Score#123
frazane wants to merge 5 commits into
mainfrom
energy-nan-handling

Conversation

@frazane

@frazane frazane commented Jun 5, 2026

Copy link
Copy Markdown
Owner

Brings nan_policy support to the multivariate Energy Score family, following the same approach introduced for the ensemble CRPS in #118. This is the first multivariate metric to gain NaN handling; the pattern is meant to carry over to the remaining multivariate scores.

As with CRPS, a member is treated as invalid if any of its variables (or its weight) is NaN, and nan_policy accepts propagate (default), raise, and omit. The omit policy zero-weights invalid members and routes the score through the existing weighted estimators, so it reduces to wiring rather than new numerics. NaN in the observations always propagates.

What changed

  • Added nan_policy to es_ensemble, twes_ensemble, owes_ensemble and vres_ensemble.
  • Reworked the apply_nan_policy_ens_mv helper (added in NaN-handling for ensemble CRPS #118 but unused and untested) to mirror the univariate apply_nan_policy_ens_uv: it now folds invalid members into the ensemble weights and returns the aligned ens_w.
  • Fixed two pre-existing bugs in the weighted numba energy kernels, both only reachable once ens_w is actually exercised on the numba backend (which the omit path now does): a scalar-indexing error (ow = ow[0]) in the outcome-weighted and vertically-rescaled kernels, and a wrong normalisation (np.mean(fw) instead of np.sum(fw * ens_w)) in the outcome-weighted kernel.
  • Added tests covering all three policies, including per-row equivalence of omit against the score computed over the surviving members (for both the nrg and fair estimators), NaN weights, and the weighted-ens_w paths that the kernel fixes depend on.

Notes

  • omit is not implemented for the akr and akr_circperm estimators (they are roll/permutation-based and cannot be expressed through member weighting), matching how CRPS treats its akr/int estimators. They raise NotImplementedError.
  • The same ow = ow[0] pattern still exists in the weighted multivariate kernel-score gufuncs (core/kernels/_gufuncs_w.py); left as a follow-up since it is outside the Energy Score scope here.

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR adds nan_policy handling to the multivariate Energy Score family (ES/twES/owES/vrES), mirroring the existing CRPS NaN-handling approach. It wires NaN member omission through the existing weighted estimators (by zero-weighting invalid members) and fixes two weighted numba energy kernels that become exercised by the new omit path.

Changes:

  • Added a nan_policy parameter to es_ensemble, twes_ensemble, owes_ensemble, and vres_ensemble, and integrated it into the scoring flow (including proper interaction with ens_w).
  • Reworked apply_nan_policy_ens_mv to align/mask ens_w and to support propagate / raise / omit consistently with the univariate helper.
  • Fixed weighted numba energy gufunc issues (scalar indexing of ow, and wbar normalisation for outcome-weighted energy) and added comprehensive tests across policies and weight scenarios.

Reviewed changes

Copilot reviewed 5 out of 5 changed files in this pull request and generated no comments.

Show a summary per file
File Description
tests/test_wenergy.py Adds NaN-policy tests for owES/vrES/twES and a uniform-ens_w equivalence guard for the weighted kernels.
tests/test_energy.py Adds NaN-policy coverage for es_ensemble, including NaNs in forecasts, NaNs in ens_w, and estimator-specific omit behavior.
scoringrules/core/utils.py Reworks apply_nan_policy_ens_mv to return aligned ens_w and implement omit-by-zero-weighting for invalid members.
scoringrules/core/energy/_gufuncs_w.py Fixes weighted numba ow/vr energy gufunc bugs (scalar indexing and wbar computation).
scoringrules/_energy.py Threads nan_policy through public Energy Score APIs and routes omit cases through the weighted code paths.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

@frazane frazane mentioned this pull request Jun 6, 2026
4 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants