Repository navigation
Flaky: wall-clock-bound tests fail under CI load (regex_redos, many_move_statements_stay_linear) #4477
Description
tests/cobol_mainframe/test_tool_regex_redos.py::test_bounded_heavy_linear_pattern_is_not_an_offender failed in CI on #4471 (run 37331076568). The PR adds no regex, and the test passed 3/3 locally under heavy-run.sh. It measures a hard-coded pattern with a timing threshold, so it is flaky under CI load. Make it deterministic (count steps or use a generous relative threshold) rather than wall-clock bound. Don't just raise the limit blindly.
🤖 Generated with Claude Code
Activity
squid-protocol commentedon Oct 5, 2026
Second wall-clock flake:
tests/core_engine/test_cics_resources.py::test_many_move_statements_stay_linearfailed on #4471 in CI at 2.03s against a 2.0s limit (run 37338266687). Locally it takes 0.32–0.34s on both main and the PR branch (3 runs each), so it's runner load, not a regression. Fix both tests the same way: a deterministic measure (operation counts, or a ratio between two input sizes to prove linearity) instead of an absolute wall-clock bound.[-]Flaky: test_tool_regex_redos bounded-heavy-linear test is wall-clock bound and fails under CI load[/-][+]Flaky: wall-clock-bound tests fail under CI load (regex_redos, many_move_statements_stay_linear)[/+]tests: ratio-based linearity check, posix census key, JSON-escaped pa…
squid-protocol commentedon Oct 6, 2026
Another instance on main, 2026-10-06 (run 37487584430, windows-latest 3.12 only; 3.10/3.11 passed on the same SHA 1797570):
test_tool_regex_redos.py::test_no_new_offendersflaggedcobol_graveyard_finder.py:112as quadratic (8.8x time for 4x input). That regex has not changed since 2026-10-04 (#4291). The growth-ratio check is wall-clock bound, so it misfires on a loaded runner, the same class as this issue.🤖 Generated with Claude Code
squid-protocol commentedon Oct 6, 2026
Third sighting, 2026-10-06: a local
pr_gatessuite run for PR #4580 under multi-agent load flaggedcobol_jcl_forge.py,cobol_to_java_uow_forge.pyandproof_reach.pyintest_tool_regex_redos.py::test_no_new_offenders. The PR touches none of them, and it now misfires locally as well as on CI. This costs agents a gate rerun per PR, so it's worth fixing soon.🤖 Generated with Claude Code
CI shepherd: ci_digest (triage + error lines + local repro) and ci_sh…
ci_digest: suite part names get the pytest repro; test_prism_suppress…
squid-protocol commentedon Oct 10, 2026
Another one in this family:
tests/core_engine/test_prism.py::test_prism_suppression_regex_bombfailed full-suite part 2/3 on #4830 (2026-10-10) at 1.006 s against its 1.0 s bound, on a loaded runner. The CI digest/shepherd (#4790) now lists it with the other wall-clock tests as a known flake, so it gets one rerun instead of a "real failure" note. The real fix here (a relative bound, or a step-count/complexity assertion instead of wall time) still applies to all three.🤖 Generated with Claude Code
ci shepherd: main's own failures labelled main; non-required infra fa…
Python 3.10 + Windows: fix the Full Suite Gate legs red on main (Part…