Summary
The current pytest suite contains tests that either are not collected by pytest at all, or are collected but do not make meaningful assertions. These files can make the suite look broader than it really is.
Examples
python/tests/unit/test_cap2weight.py defines run_tests() and only executes it under if __name__ == "__main__", so pytest collects zero tests from the file.
- Many comparison wrappers under
python/tests/compare/ contain no direct assertions and rely entirely on the shared runner.
- Many integration tests under
python/tests/integration/ write CSV/PNG artifacts but contain no assert statements or np.testing checks.
python/tests/unit/calibration/test_verify_estimate_frequencies.py has pytest test functions but currently only prints pass/bad information; the intended accuracy threshold assertions are commented out.
Why this matters
- Regressions can pass if a test prints a failure-like message but never asserts.
- Files may appear to be part of the pytest suite while contributing no collected tests.
- The suite is harder to trust as a CI signal because it mixes smoke scripts, generators, and actual assertions.
Suggested fixes
- Convert
test_cap2weight.py into normal pytest functions with explicit assert/np.testing checks.
- Add real assertions to integration tests that currently only generate artifacts.
- In frequency-estimation tests, turn the printed threshold checks into actual assertions.
- Keep comparison wrapper assertions centralized if desired, but document that the wrapper is intentionally assertion-free and relies on
run_comparison_suite() raising failures.
Verification target
A static audit should find no collected test_* functions with zero assertion-like behavior unless they are explicitly documented as runner wrappers.
Summary
The current pytest suite contains tests that either are not collected by pytest at all, or are collected but do not make meaningful assertions. These files can make the suite look broader than it really is.
Examples
python/tests/unit/test_cap2weight.pydefinesrun_tests()and only executes it underif __name__ == "__main__", so pytest collects zero tests from the file.python/tests/compare/contain no direct assertions and rely entirely on the shared runner.python/tests/integration/write CSV/PNG artifacts but contain noassertstatements ornp.testingchecks.python/tests/unit/calibration/test_verify_estimate_frequencies.pyhas pytest test functions but currently only prints pass/bad information; the intended accuracy threshold assertions are commented out.Why this matters
Suggested fixes
test_cap2weight.pyinto normal pytest functions with explicitassert/np.testingchecks.run_comparison_suite()raising failures.Verification target
A static audit should find no collected
test_*functions with zero assertion-like behavior unless they are explicitly documented as runner wrappers.