Skip to content

Return NaN for non-finite input instead of a made-up entropy - #9

Merged
Par-python merged 1 commit into
masterfrom
fix/nan-handling
Sep 21, 2026
Merged

Par-python merged 1 commit into
masterfrom
fix/nan-handling

Conversation

@Par-python

Copy link
Copy Markdown
Owner

Problem

A single NaN or ±inf in the input produced a plausible-looking entropy instead of NaN:

x = np.random.default_rng(0).standard_normal(64); x[10] = np.nan
permutation.compute(x)                    # 2.5336 (argsort ranks NaN last, counted as a pattern)
spectral.compute(x)                       # -0.0   (NaN spectrum; the psd > 0 filter keeps nothing)
permutation.compute(np.full(64, np.nan))  # -0.0

rolling inherited this, so one gap in a logged metric silently corrupted every window that contained it. It was found while building training-early-warning, where spectral entropy dropped to exactly 0 in a window that ran past a NaN loss.

Fix

  • _core.nan_on_non_finite wraps every measure's _kernel: compute, normalized, rolling and delta return NaN when the input or window holds NaN or ±inf.
  • rolling is NaN for exactly the windows containing the gap; every other window is unchanged.
  • The same guard covers the paths that bypass the kernels: multiscale.compute, transfer, spectral.normalized, and divergence.kl / js.
  • Zero entropy is returned as 0.0, never -0.0.
  • README documents the missing-value behavior and that rolling is causal. CHANGELOG marks the change as one that changes results.

EntropyFeatures is unchanged: scikit-learn's check_array already rejects NaN input with a clear ValueError, as every sklearn estimator does.

Tests

tests/test_missing_values.py: NaN, inf and -inf for every single-series measure, all-NaN input, rolling windows NaN only around the gap and identical elsewhere, delta, multiscale, transfer (both estimators, gap in x or y, rolling), divergence, and positive zero. The full suite passes locally (248 tests).

Local ruff is 0.15.15 while CI pins 0.16.7, so it's worth watching the lint job.

🤖 Generated with Claude Code

One NaN or inf in the input used to produce a plausible-looking number:
permutation.compute counted NaN as an ordinal pattern (argsort ranks it
last), spectral.compute returned -0.0 because a NaN spectrum filters to
nothing, sample.compute returned a large finite value, and shannon.compute
raised. rolling inherited this, so one gap silently corrupted every window
that contained it.

Every measure's kernel is now wrapped by _core.nan_on_non_finite, so
compute, normalized, rolling and delta return NaN for exactly the windows
that hold a non-finite value and leave the rest of the series unchanged.
multiscale, transfer, spectral.normalized and divergence.kl/js get the
same guard. Zero entropy is also returned as 0.0, never -0.0.

Found while building the training early-warning study
(github.com/Par-python/training-early-warning).

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@Par-python
Par-python merged commit 6f3ead1 into master Sep 21, 2026
9 checks passed
@Par-python Par-python mentioned this pull request Sep 21, 2026
3 of 4 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant