Skip to content

ci: reduce repeated verification overhead - #106

Merged
noeltock merged 2 commits into
mainfrom
codex/ci-performance-0915
Sep 15, 2026
Merged

noeltock merged 2 commits into
mainfrom
codex/ci-performance-0915

Conversation

@noeltock

@noeltock noeltock commented Sep 15, 2026 •

Copy link
Copy Markdown
Member

Problem

A README or demo update pushed to main currently triggers the full CI matrix. Sequential WordPress proof cases also stop the environment after each run, so the next case has to start it again. The WordPress job took 17m 57s in the v0.9.6 CI run, including 13m 45s for automated acceptance.

Closes #105.

Solution

Same check v0.9.6 baseline This revision Observed reduction
WordPress job 17m 57s 12m 10s 5m 47s (32%)
Automated acceptance step 13m 45s 7m 31s 6m 14s (45%)
Real-WordPress test file, 8 tests 777.212s 431.972s 345.240s (44%)

These are observations from one before/after GitHub run, not a controlled attribution of every saved second. Runner variability and other step timings differ. Docs/demo routing avoids irrelevant work; its savings are separate and are not included in this comparison.

Use the existing conservative path classifier for main pushes as well as PRs, and keep WordPress running between sequential proof cases. Each proof still stages and installs its exact ZIP, runs its browser assertions, and writes its own receipt. The suite stops the environment at the end only if it was not already running before the suite began.

Behavior that changes or must stay true Read first Proof
Docs/demo-only main pushes take the focused route .github/workflows/ci.yml, scripts/ci-scope.mjs Classifier tests and executed PR/push event-range checks
Runtime, dependency, deleted, renamed and unknown paths require full checks scripts/ci-scope.mjs Conservative-route regression tests
Each WordPress case retains its exact artifact and assertions dev/test/proof-real-wordpress.test.ts Existing staged-ZIP tests; full WordPress CI: 9 tests passed
Cleanup preserves pre-existing environments and refuses unknown status dev/test/proof-wordpress-lifecycle.ts Lifecycle tests covering ownership and status-probe failures
Required jobs cannot disappear behind a skipped result .github/workflows/ci.yml, ci-scope job Executed aggregator checks for all three routes

Diff

+192 −21 · 6 files · no public API change

Simplified control flow:

 CI event
   pull request → classify base..head
-  main push    → full matrix
+  main push    → classify before..after
+    allowlisted added/modified docs or demo → focused contracts
+    runtime, dependencies, deleted/renamed/unknown paths → full matrix
+    missing or zero boundary → full matrix
+    unavailable diff → fail classification

 Sequential WordPress proof suite
+  beforeAll → inspect status; record cleanup ownership
   for each proof
     start environment
     stage and install exact SHA-named ZIP
     run browser assertions and write receipt
-    stop environment
+    keep environment running
+  afterAll → stop only a suite-owned environment, including after failure

The release workflow keeps its full checks. The existing workflow-level final cleanup remains in place.

Testing & verification

Reviewed revision: 57f15167ea5ec762dd1e5e5660a8334557d8aca7 · Environment: local macOS checkout; GitHub Ubuntu CI passed on Node 20.19.0, 22.13.0 and 24.0.0.

  • node --test dev/test/ci-scope.test.mjs: 9 passed.
  • npx vitest run dev/test/proof-wordpress-lifecycle.test.ts dev/test/proof-control-setup.test.ts: 11 passed.
  • npx --no-install vitest run dev/test/skill.test.ts dev/test/node-support.test.ts: 9 passed.
  • npm run typecheck: passed.
  • git diff --check: passed.
  • Workflow YAML parsed successfully; demo shell syntax and nonempty static-asset checks passed.
  • Executed the workflow classifier shell against historical docs-only PR/push ranges, a zero boundary and an unavailable base: expected routes/failure in all four cases.
  • Executed the final CI aggregator shell for docs, skill and full routes: each accepted required successes and rejected a skipped selected job.

Full CI run 34913012901 passed: all three Node versions, packed consumers, WordPress proof and final CI scope gate. WordPress retained all 9 tests across 2 files, including all 8 real-WordPress proof cases.

Not verified: the reduced route has not yet run as a live main-push event; its exact workflow shell was exercised locally. No repeated performance sample has been collected.

Risk / rollout

Retaining WordPress could expose state left by an earlier proof. Exact ZIP installation and all existing browser assertions remain the acceptance gate. Routing could skip required checks if classification were incomplete, so unknown changes select full checks, unavailable diffs fail, and the final CI scope job validates the selected route.

  • Focused routing and lifecycle checks pass.
  • Full CI passes on the reviewed revision.
  • Record WordPress job and acceptance-step timings against the baseline.

Detection: existing WordPress assertions, artifact receipts and the final CI scope check. Rollback: revert 57f1516 and 9e5694f to restore the previous lifecycle and routing. No package release or data migration is required.

Authored by: Codex (GPT-6).

@noeltock
noeltock merged commit b207280 into main Sep 15, 2026
9 checks passed
@noeltock
noeltock deleted the codex/ci-performance-0915 branch September 15, 2026 00:39
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

CI: reduce repeated verification overhead

1 participant