Skip to content

Retry a screenshot Chromium refuses for a moment, and fix three CI flakes - #44

Merged
brynary merged 1 commit into
robustnessfrom
ci-flakes
Oct 5, 2026
Merged

brynary merged 1 commit into
robustnessfrom
ci-flakes

Conversation

@brynary

@brynary brynary commented Sep 29, 2026

Copy link
Copy Markdown
Contributor

Stacked on #42. Fixes the three tests that each failed once in CI after the #43 fix went up the stack, and passed on re-run.

Two snapshot tests: a real robustness fix

snapshot_count_percent_threshold_and_dimension_boundaries (Ubuntu, #36) and a_snapshot_mismatch_fails_and_writes_actual_and_diff (macOS, #42) both failed with:

error: internal: page.screenshot: Protocol error (Page.captureScreenshot): Unable to capture screenshot

Chromium refused a capture for a moment on a busy runner, and Whirl reported a runtime error. A user's SNAPSHOT could fail in CI the same way, so the fix is in the shim, not the tests: SNAPSHOT (page, viewport, and element captures), SCREENSHOT, and JUDGE now take such a capture again after a short pause, until the step's time runs out. Any other capture error fails as before. SPEC section 7 says so, and a shim unit test covers the retry, an unrelated error, and an expired deadline. No protocol change.

The 60 fps video test: a machine-dependent threshold

chromium_video_records_at_60_fps_by_default_and_at_the_requested_rate (macOS, #35) failed with an animating page should record distinct frames, got 61 of 128. On a busy runner, with --trace sharing the screencast, Chrome supplies fewer distinct frames, and the recorder repeats the last one to keep 60 fps, as designed. The check exists to show that the animation was recorded; a still page gives one or two distinct frames. Both rate checks now require a quarter of the frames to differ instead of half, which still separates animation from a frozen page by a wide margin.

Validation: mise run check; CI below.

🤖 Generated with Claude Code

…akes

Two snapshot tests failed once in CI with "Page.captureScreenshot:
Unable to capture screenshot": Chromium refused a capture for a moment
on a busy runner, and Whirl reported a runtime error. SNAPSHOT (page,
viewport, and element captures), SCREENSHOT, and JUDGE now take such a
capture again after a short pause, until the step's time runs out. Any
other capture error fails as before.

The 60 fps video test failed once on macOS with 61 distinct frames of
128: on a busy runner, with --trace sharing the screencast, Chrome
supplies fewer distinct frames and the recorder repeats the last one to
keep the rate. The test only needs to show that the animation was
recorded, and a still page gives one or two distinct frames, so both
rate checks now require a quarter of the frames to differ instead of
half.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@brynary
brynary added this pull request to stack #49 October 1, 2026 20:57
@brynary
brynary merged commit 55320f0 into mock Oct 5, 2026
9 checks passed
@brynary
brynary deleted the ci-flakes branch October 5, 2026 18:29
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant