Repository navigation
Stop the watchdog before a parked run resumes its step - #121
Merged
Merged
Conversation
The two watchdog integration tests prove that a pause or a pending question parks the stall clock: the run waits for three budgets with nothing running and is not cancelled. Both then let the run continue into a host step with the watchdog still armed, so the step's sandbox launch was measured against the budget too. On a loaded CI runner that launch took longer than the 400 ms budget and the pause test tripped, on main as well as on unrelated branches. Now each test stops the watchdog before it unparks the run: the pause test before its unpause, the question test before its answer. The three parked budgets remain the proof. An unpause restarting the full budget is covered by the watchdog's own unit test with paused time. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Fixes the flaky
a_pause_parks_the_watchdogtest, and the same shape ina_pending_question_parks_the_watchdog, incrates/petri/lib/tests/controls.rs.Why
Main's CI is red on
a_pause_parks_the_watchdogafter #117, and the same test failed main on October 7 after #78. It has also tripped locally under full parallel load. Neither change touched the watchdog or the controls.The test pauses a run with a 400 ms stall budget, waits 1.2 s, then unpauses. The unpause restarts the full budget, and the run's next action is a host step, which launches the host plugin and creates a sandbox. The CI output shows the plugin starting 60 ms after the unpause and the test ending with the watchdog tripped 850 ms later: the launch took longer than the budget with no activity record in between. That is wall-clock latency on a loaded runner, not idleness, and not what the test is proving.
The question test has the same shape with a 1 s budget. It has not been seen failing, but it would under the same conditions.
Fix
Each test stops the watchdog before it unparks the run: the pause test before its unpause, the question test before its answer. The three parked budgets with nothing running remain the proof that a pause or a pending question parks the clock. An unpause restarting the full budget is covered by the watchdog's own unit test, which runs under paused time.
No engine code changes.
Verification
mise run fmt:check,cargo clippy -p petri --all-targets --all-features -D warnings🤖 Generated with Claude Code