Skip to content

Further Improve robustness of model classifier tests#4450

Merged
alejoe91 merged 1 commit into
SpikeInterface:mainfrom
chrishalcrow:further-improve-robustness-unitrefine-tests
Mar 17, 2026
Merged

Further Improve robustness of model classifier tests#4450
alejoe91 merged 1 commit into
SpikeInterface:mainfrom
chrishalcrow:further-improve-robustness-unitrefine-tests

Conversation

@chrishalcrow

Copy link
Copy Markdown
Member

After merging (#4447), there were still flakey failures in the tests.

This PRs makes the training data more robust in two ways:

  1. The number of units in the generated recording has been reduces. This is better because the units are spaced out with some minimum distance between each one. If the number is too high, a unit can be produced quite far from the probe and hence have a small amplitude.
  2. We give the model repeated data, ensuring it sees all the data in its training. Hence it really should not make a mistake. You don't want to do this when actually training a model, but good for testing.

When testing before, I used pytests count flag. This was stupid: the test was using a cache so wasn't retrained each time. This time I ran the test repeatedly from scratch using something like

for i in {1..1000}; do echo "--- Run #$i ---"; pytest test_file.py::test_name -q || break; rm -r path_to_cache; done

Thanks Gemini!

No failures on 1000 runs :)

@chrishalcrow chrishalcrow added the testing Related to test routines label Mar 16, 2026
@alejoe91 alejoe91 merged commit bb7c7ad into SpikeInterface:main Mar 17, 2026
15 checks passed
@chrishalcrow chrishalcrow deleted the further-improve-robustness-unitrefine-tests branch March 17, 2026 08:15
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

testing Related to test routines

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants