fix(tests): publish a readiness marker atomically, and place tests by cost - #37
Merged
Merged
Conversation
… cost The process-death fixture created its readiness marker and then filled it, while the parent took existence for completeness and parsed the contents, so it read an empty file and reported a wrong marker. It failed three runs in four. Writing beside the destination and renaming makes existence imply completeness, and the suite has now run clean twenty times. Stage 31 had recorded this as a Tier 4 test to relegate to its own lane. Measuring first refuted that: the test costs 50 ms of a 15.77 s workspace suite, and its permission-store sibling costs 4.16 s spawning processes the same way without anyone mentioning it. Nothing needed relegating, and CTL-1, COL-4, COL-5, CHB-2 and CHB-3 have real proof again across three evidence documents. So the standard places a test by cost and determinism rather than by mechanism, which had condemned the cheap deterministic one and overlooked the expensive one. It also states the handshake rule this bug taught, beside the readiness-signal rule that already governed the pattern. Both the Tier 4 rule and the flaky rule stopped pointing at a lane CI does not have: verify runs cargo test and all seven smoke scripts in one job, so there was never a slower lane to move anything into. A flaky test is fixed or deleted with its claim marked unproven. Of 40 script citations in evidence tables, 36 already named what the script proves. The four that listed a script or a gate among test function names now say which kind of evidence it is, so a reader can tell how firmly the invariant stands. The standard remains over its advisory byte budget. The paragraphs explaining why the policy changed are the part that gets deleted once the corpus has settled into it.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Stage 31. One test-fixture fix, and the standard corrections it taught.
Why
provisioning_process_death_recovers_one_exact_passive_childfailed three runs in four. The fixturecreated its readiness marker and then filled it, while the parent took existence for completeness
and parsed the contents — so it read an empty file and reported a wrong marker. Writing beside the
destination and renaming makes existence imply completeness. The suite has now run clean twenty
times.
Measuring refuted two of the three things stage 31 claimed
Stage 31 was written in this repo by an agent, and it was wrong twice.
The rule is not "don't spawn processes". It is: a marker whose existence is the signal must not
also carry data the reader parses — unless it is published atomically. Two sibling tests use the
same spawn-a-child pattern and are safe for structural reasons: one waits for full exit, the other's
marker is a fixed literal nobody parses.
What changed
standards/testing.md— two corrections and one addition:50 ms deterministic process test and overlooked a 4.16 s one.
verifyrunscargo test --workspaceand all seven smoke scripts in one job, so there was never a slowerlane to move anything into. A flaky test is fixed, or deleted with its claim marked unproven.
pattern.
Four evidence rows now match the convention the other 36 use: named tests first, then
;and aclause saying what a script or a gate proves —
composer-menu.mdSKP-1,selection-and-copy.mdSEL-5,
transcript-entry.mdENT-1,session-journal.mdJRN-5. CTL-1, COL-4, COL-5, CHB-2 and CHB-3have real proof again across three evidence documents.
Stage 31, the phase status and the roadmap row record what happened rather than what was
predicted: fixed, not moved.
Not done here, deliberately
plexmaton-command/src/prefix.rscarriesMAX_PARSE_TIME = 20 msas a wall-clock bail-out on everycommand admission. When it trips, the parse is abandoned and a prefix Deny stops matching, so
evaluation falls through to grants where an exact grant can allow the call — a Deny skipped because
the machine was busy. Observed as 1 failure in 5
cargo test --workspaceruns.That belongs to
spike/permission-boundary, whose head commit is already "fence a command's writesinstead of inspecting it" — fixing it here would be thrown away and would conflict.
Blast zone verified disjoint: none of the 8 files here appears among that branch's 14
(
git diff --stat main...spike/permission-boundary).Verification
All run outside the sandbox — 17 runtime tests bind loopback fixture servers, which a sandbox
refuses, and that reads as 17 failures.
cargo test --workspace --lockedexit 0cargo clippy --workspace --all-targets --locked -- -D warningsexit 0standards/testing.mdis 274 bytes larger than at branch start, not smaller as planned: it gained arule it did not have. Budgets are advisory, and the paragraphs explaining why the policy changed are
what gets deleted once the corpus has settled into it.