Repository navigation
Conversation
… including on a reused tree
…_TEST_TIMEOUT does not stall them bats 1.13 runs its timeout watcher as a child of the test shell, so a bare wait in a test body waits for the watcher and holds the test until the limit. The storage fan-outs and the sync autostart race now wait on the PIDs they started, and the nested bats run in test_spawn no longer inherits the timeout.
…s timeout watcher A test that defines kill() as a no-op leaves it in the shell bats runs the test in, and bats uses that same kill to stop its BATS_TEST_TIMEOUT watcher at the end of the test. The watcher then ran its whole 180 s, so files that stub kill (teardown_reap, instance_id, resolve_project, delivery, ...) were held for minutes and pushed macOS shards past the 30 minute cap. teardown_test_env now drops those function stubs first, and the teardown_reap stubs live in a subshell.
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Two changes to
tests.ymlso a run stops waiting on things that are not tests:BATS_TEST_TIMEOUT=180on the shard job; bats 1.13, the versionnpm install -g batsinstalls, supports it). A single hung test used to hold its shard until the 30-minute job cap:test_remoteAdd event log schema to SQLite driver with backward-compat reads #54 sat for 30 minutes, and the rerun on top took the run to 65-87 minutes. Now the one test fails after 180 s, is named in the log, and the shard goes on. No job cap is raised.app typecheckis also a macOS job but lasts ~10 s when nothing inapp/changed; it is not one of the 5.) GitHub runs 5 macOS jobs at a time for this account. Run 43816536201 asked for 7 (6 bats shards plus the macOS storage contract;app typecheckalso takes a macOS slot for ~10 s): five started, bats 4/6 and the storage leg waited 17-19 minutes for a slot, and the run took two waves.total), and the aggregatebatsjob re-checks coverage for each (SHARD_TOTALS: '10 5').The storage step keeps the gate its job had:
contracts_neededalone, not a reused tree (a tree a PR run recorded green without running the contract is pushed withcontracts_needed=true, and the reuse must not skip it), and bats is installed for it even when the suite itself is skipped.Required checks
Branch protection requires
bats(the aggregate) andbats (windows-latest, install helpers). Neither is renamed. Shard check names change frombats (<os> n/6)tobats (<os> n/<total>), andstorage contract (jsonl, macos-latest)goes away; none of those is a required context (read from the branch protection settings).Planned shard seconds (re-measured table, macOS seconds)
Shard 5 also carries the storage contract (~2 min plus setup). Five macOS shards are each ~19% heavier than six were, but all five start at once; six shards plus the storage leg took two waves. The 30-minute shard cap is unchanged, so the macOS margin is thinner (see below).
Before / after (job durations, minutes)
Before: run 43816536201 (macOS shards 19-20 min, 4/6 started 18 min late and finished at 26 min; whole run 26 min). After: this PR's run (filled in below).
Longest single test (measured)
From the per-test timestamps of run 43816536201 (macOS 2932 tests, Ubuntu 3403): longest macOS test 59.9 s (
test_remote_sync#54, 10,000-path roster sweep), then 54.3 s (test_agmsgd_daemon_sh#15), 53.5 s (test_install#106), 46.9 s (test_watch#1), 42.1 s (test_codex_bridge_launcher#8). Longest Ubuntu test 39.7 s. 59.9 s x 1.5 = 90 s, under the 180 s cap, so no passing test is expected to hit it; no test needs splitting for this.Risk
macOS runner speed varied by up to 1.35x between runs in earlier measurements, so a 1414 s plan can reach ~32 minutes on a slow runner, past the 30-minute cap. The 180 s per-test cap removes the hang case, not slowness. If a macOS shard hits the cap in practice, the fix is to run fewer macOS tests per PR (next change), not a longer cap.
Process spawns
No per-test or per-tick spawns are added. One extra step on macOS shard 5 (
batson one file, already run by the old job). Onebatstimeout watcher per test is bats' own (a backgroundsleep).Tests
tests/test_ci_sharding.batsnow checks the per-OS totals against the matrix, the excluded macOS shards, the 5-job macOS limit andSHARD_TOTALS;tests/test_ci_workflow.batsfollows the new shard name and the new step. The fourcheck-*.shgates and the private-name check pass.