Return NaN for non-finite input instead of a made-up entropy - #9
Merged
Merged
Conversation
One NaN or inf in the input used to produce a plausible-looking number: permutation.compute counted NaN as an ordinal pattern (argsort ranks it last), spectral.compute returned -0.0 because a NaN spectrum filters to nothing, sample.compute returned a large finite value, and shannon.compute raised. rolling inherited this, so one gap silently corrupted every window that contained it. Every measure's kernel is now wrapped by _core.nan_on_non_finite, so compute, normalized, rolling and delta return NaN for exactly the windows that hold a non-finite value and leave the rest of the series unchanged. multiscale, transfer, spectral.normalized and divergence.kl/js get the same guard. Zero entropy is also returned as 0.0, never -0.0. Found while building the training early-warning study (github.com/Par-python/training-early-warning). Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
A single NaN or ±inf in the input produced a plausible-looking entropy instead of NaN:
rollinginherited this, so one gap in a logged metric silently corrupted every window that contained it. It was found while building training-early-warning, where spectral entropy dropped to exactly 0 in a window that ran past a NaN loss.Fix
_core.nan_on_non_finitewraps every measure's_kernel:compute,normalized,rollinganddeltareturn NaN when the input or window holds NaN or ±inf.rollingis NaN for exactly the windows containing the gap; every other window is unchanged.multiscale.compute,transfer,spectral.normalized, anddivergence.kl/js.0.0, never-0.0.rollingis causal. CHANGELOG marks the change as one that changes results.EntropyFeaturesis unchanged: scikit-learn'scheck_arrayalready rejects NaN input with a clearValueError, as every sklearn estimator does.Tests
tests/test_missing_values.py: NaN, inf and -inf for every single-series measure, all-NaN input, rolling windows NaN only around the gap and identical elsewhere,delta,multiscale,transfer(both estimators, gap in x or y, rolling),divergence, and positive zero. The full suite passes locally (248 tests).Local ruff is 0.15.15 while CI pins 0.16.7, so it's worth watching the lint job.
🤖 Generated with Claude Code