Repository navigation
ci(efficiency): reduce redundant cost and latency without weakening required quality/security gates #550
Description
#1017 reviewer signal / latency measurement — 2026-10-09
Codex review is delivering unique, material P1/P2 findings on R-15 conversion: three initial threads resolved on the previous head and a fourth P1 cursor invalidation after restart on ca26d16. Source/merge operator must fix these, but redundant reviewer triggers, quota-limited CodeRabbit/Sourcery and unrelated #780/#931 automation must not introduce a new daily merge delay. Measure final-head end-to-end wait, queue/runner segments, duplicate CI trigger cost and per-provider unique actionable findings before governance redesign. Do not suppress required deterministic CI/security or material review. This owner remains P2 efficiency; #510 trusted-execution security and #764 mutation evidence remain distinct owners.
Purpose
Create one bounded S12 owner for CI efficiency, latency and duplication. The repository already has strong granular quality/governance trackers; what is missing is an integration-level optimization contract that decides where each expensive check belongs, how often it needs to run, and how to reduce repeated work without weakening evidence.
This issue is explicitly about execution efficiency. It must not become a pretext to lower thresholds, remove coverage/security checks, or bypass branch protection.
Existing specialist owners to reconcile, not duplicate
- test(ci): expand coverage scope to executable api, workers, and contexts #446 — executable coverage scope and ratchet graduation;
- ci(performance): make bundle budgets evidence-based regression budgets #450 — evidence-based bundle budgets;
- workflow-policy-check: compound if: expressions + cross-workflow gate coverage #506 — workflow-policy parser/cross-workflow coverage gaps;
- ci: write-scoped governance jobs (pr-size, workflow-policy) enforce from PR-controlled code, not a trusted ref #510 — trusted execution boundary for write-scoped governance jobs;
- tooling: define hermetic exact-tree validation / compiler sandbox boundary #504 — hermetic exact-tree validation architecture question;
- native: promote macos-15-intel qualification lane into the release matrix #507 — Intel macOS qualification/release-cost decision;
- supply-chain: document trust-policy gaps across Docker/GHCR, Tauri release artifacts, and pnpm dependency provenance #529 — supply-chain/provenance checks where CI is part of enforcement.
This issue owns the whole-pipeline cost/latency model and de-duplication plan, not those specialist semantics.
Inventory first
For every workflow/job on main, pull requests, tags, schedules and manual dispatch, record:
- trigger;
- required vs advisory status;
- runner/OS/runtime matrix;
- median/p95 queue time and execution time where observable;
- duplicated setup/build/test work;
- cache inputs/outputs and hit assumptions;
- artifact production/consumption;
- whether result is needed for every PR, only affected paths, main, release, schedule, or evidence cadence;
- security/trust reason a job cannot safely reuse PR-controlled artifacts;
- failure signal uniqueness (what defect class only this job catches).
Optimization principles
- Preserve evidence before reducing frequency. A slow check that uniquely catches a real class of failures is not redundant merely because another green job looks similar.
- Share deterministic artifacts only across compatible trust boundaries. Do not let untrusted PR artifacts become authority for write-scoped/release/security jobs.
- Prefer one build, many consumers when platform/runtime semantics genuinely match.
- Use path/change awareness only when sound. Shared configuration, generated files, workspace packages and transitive dependencies must invalidate affected gates correctly.
- Separate fast merge feedback from deep evidence cadence. Expensive mutation/soak/nightly/browser/platform work can be scheduled if merge-safety coverage remains adequate.
- Do not optimize by weakening thresholds, skipping required matrices, or hiding flaky failures.
Specific audit targets
- duplicate dependency installation/setup across jobs;
- repeated production builds that can safely consume one immutable artifact;
- repeated i18n/docs/parity/governance scans with identical inputs;
- Node 22/24 matrix value versus duplicated work, preserving compatibility evidence;
- Storybook/VRT/Lighthouse execution topology;
- Playwright vs Deep E2E overlap and unique scenarios;
- CodeQL/Semgrep/OSV/Socket/GitGuardian boundaries and trigger cadence;
- Tauri platform builds versus qualification/release-only lanes;
- cache-key breadth and stale-cache correctness;
- concurrency cancellation of superseded PR runs;
- generated artifact reuse and retention;
- merge-queue/main/tag duplicate cycles;
- reviewer/provider integrations that trigger repeatedly on correction commits but add little new signal.
Required measurements
Before and after each optimization, record at minimum:
PR feedback median/p95
full required-check wall clock
runner minutes by OS/runtime
cache hit/miss where measurable
number of duplicated build/install operations
failure-detection coverage retained
Cost optimization should be expressed as evidence, not intuition.
Safety gates
Any proposed optimization must answer:
- Can a changed file influence a skipped job indirectly?
- Does artifact reuse cross a trust boundary?
- Could cancellation leave an old/stale result satisfying a required context?
- Does a matrix reduction remove a supported runtime/platform oracle?
- Does cache reuse hide dependency/config drift?
- Does the change weaken test(ci): expand coverage scope to executable api, workers, and contexts #446/ci(performance): make bundle budgets evidence-based regression budgets #450/workflow-policy-check: compound if: expressions + cross-workflow gate coverage #506/ci: write-scoped governance jobs (pr-size, workflow-policy) enforce from PR-controlled code, not a trusted ref #510 or supply-chain guarantees?
If uncertain, preserve the stronger current behavior until deterministic proof exists.
Acceptance criteria
- A current workflow/job cost and latency inventory exists.
- Every required check has a documented unique purpose or is consolidated safely.
- Duplicate setup/build/test work is reduced where trust/input semantics allow it.
- Superseded PR runs cancel safely where appropriate.
- Cache/artifact reuse has deterministic invalidation and trust-boundary rules.
- Expensive advisory/deep evidence runs have an explicit cadence rather than accidental duplication.
- Node/platform/browser matrices retain evidence-backed support coverage.
- No quality, Codecov, security, signatures, governance or branch-protection requirement is weakened merely for speed/cost.
- Before/after wall-clock and runner-minute evidence is recorded.
- Specialist follow-ups (test(ci): expand coverage scope to executable api, workers, and contexts #446/ci(performance): make bundle budgets evidence-based regression budgets #450/workflow-policy-check: compound if: expressions + cross-workflow gate coverage #506/ci: write-scoped governance jobs (pr-size, workflow-policy) enforce from PR-controlled code, not a trusted ref #510/tooling: define hermetic exact-tree validation / compiler sandbox boundary #504/native: promote macos-15-intel qualification lane into the release matrix #507/supply-chain: document trust-policy gaps across Docker/GHCR, Tauri release artifacts, and pnpm dependency provenance #529) remain individually owned and reconciled.
Non-goals
- making CI green by deleting tests;
- lowering coverage/bundle/security thresholds;
- bypassing protected-branch/ruleset requirements;
- trusting PR-controlled code/artifacts in privileged workflows;
- eliminating deep scheduled evidence merely because it is slow;
- optimizing reviewer/model quotas by suppressing genuinely useful findings.
Activity
qnbs commentedon Sep 10, 2026
Progress reconciliation — CI-performance Waves 1–2 landed
This umbrella remains open, but two concrete optimization slices are already complete and should not be rediscovered as future work:
Wave 1 — pnpm bootstrap variance: DONE via #604
pnpm/action-setup's npm-based self-install was measured as the dominant setup-variance source (roughly 20s–6m in sampled jobs while warm-store dependency install itself stayed much smaller). #604 replaced both relevant bootstrap call sites withpnpm/setup's standalone binary path while preserving the existing dependency-store cache and trust-boundary semantics.Wave 2 — Browser Quality duplicate setup: DONE via #608
The former separate VRT and Lighthouse jobs were consolidated into one
browser-qualityjob, sharing setup/build-artifact acquisition while preserving independent diagnostic execution and required/advisory failure semantics. The change reduced duplicated runner work without removing VRT or Lighthouse evidence.Remaining #550 scope
Keep this issue open for the still-unfulfilled whole-pipeline acceptance criteria: current cost/latency inventory, any further safe artifact/setup reuse, cancellation/cache analysis, matrix evidence decisions, and before/after runner-minute/critical-path measurements.
Do not repeat #604/#608, and do not optimize by weakening #446/#506/#510, browser/platform coverage, security gates, or required
✅ CI Successsemantics.qnbs commentedon Sep 12, 2026
Playwright scale-out addendum — measure first, shard before raising workers or changing frameworks
#736 now owns the required Playwright gate architecture. #550 remains the CI cost/latency/duplication authority and should supply the evidence that determines when/how that gate scales.
Preserve current runner-resource discipline
Current
playwright.config.tsdeliberately uses:Do not treat
workers: 1as an accidental inefficiency. On GitHub-hosted runners it is currently a resource-safety choice that limits concurrent browser RAM/CPU pressure and makes startup/persistence races easier to diagnose.Any proposal to raise CI workers must first measure:
Preferred first scale-out mechanism
If the required ordinary E2E suite exceeds its acceptable feedback budget, prefer Playwright-native sharding before a framework migration or high in-process worker count.
Initial candidate topology:
Do not implement sharding merely because #736 exists. First establish the trigger from measured current behavior.
Required measurements for the sharding decision
Record at minimum over a representative recent window:
Set the scale-out trigger from evidence, not a fixed test-count threshold.
Examples of possible evidence-based triggers to evaluate, not pre-authorize:
Sharding safety invariants
Any sharded implementation must prove:
ci-successcontract;Prefer Playwright-supported report/artifact merge facilities if report aggregation is needed rather than custom brittle concatenation.
Required gate stays lean even after sharding
Sharding is a latency optimization, not permission to move all deep tests into required CI.
#736 keeps the required gate focused on smoke + critical journeys. #623 continues to govern advisory promotion. #709 decides coverage disposition. #710/#711 add functional scenarios under those authorities.
Web/native cost boundary
Do not duplicate the browser suite in Tauri/Qt merely to increase coverage. Shared React behavior should remain browser-qualified once; native jobs should target only genuinely distinct filesystem/updater/native-dialog/window/IPC/packaging semantics.
Native matrix/runtime cost should be measured separately from web Playwright cost.
Additional acceptance implications for #550
workers: 1policy is retained or changed only with evidence;qnbs commentedon Sep 15, 2026
Current-main addendum — main-push concurrency still loses per-SHA diagnostic evidence — 2026-09-16
Revalidated against
main4137286887275ca03aa5b75906e6b529c68c786c.Current workflow policy is still:
docs/CI.mddescribes this as “main branch pushes are never cancelled — each push builds its own run”. That wording is too strong for GitHub concurrency semantics:cancel-in-progress: falseprevents cancellation of the currently running job, but a concurrency group permits only one pending run; a newer queued main push can replace/cancel an older pending main run before it starts.The 2026-09-16 audit observed this during the recent Dependabot burst: intermediate main SHAs such as the jsdom-30 merge and several adjacent dependency merges had their queued resulting-main runs superseded, while the final cumulative main state was tested green.
Why this matters
Final cumulative-green evidence protects current main, but it weakens per-merge causality:
That is especially undesirable for major/toolchain/security-sensitive dependency updates.
#550 decision to make
Measure and compare a small set of policies rather than changing concurrency blindly:
Account for runner-minute cost and deploy duplication. Do not simply disable all concurrency optimization without measurements.
Required truth/documentation correction
Until a stronger policy lands, current docs should not claim every main push necessarily builds its own run. State the actual GitHub pending-run behavior and distinguish:
cancel-in-progress;Add this to #550's cost/latency/evidence inventory and preserve the standing requirement that high-risk merges retain causal resulting-main evidence where practical.
qnbs commentedon Sep 16, 2026
Current-main CI-evidence addendum — burst merges can cancel queued per-SHA main runs — 2026-09-16
A fresh audit of the 2026-09-15 Dependabot merge burst found a distinct evidence/cost tradeoff that belongs in #550's cancellation/topology analysis.
The current workflow concurrency behavior can replace/cancel queued
mainpush runs when several merges land faster than the full pipeline drains, even with the intendedcancel-in-progress: falsebehavior for a run already executing. During the dependency burst, intermediate resulting-main runs for merges including #752/#753/#754/#756/#758 did not all reach terminal full-pipeline evidence before a newermainSHA became authoritative; the final cumulative main state was subsequently green.This is not evidence that those merged states were broken, and a later cumulative green main proves the final combined tree. It does mean the repository cannot always claim independent full-CI qualification for every intermediate merge SHA, which reduces bisect/attribution precision when a burst contains a breaking dependency change.
Add this to the #550 topology decision
Measure and explicitly choose the desired invariant for
main:Do not change concurrency blindly. First quantify:
A useful bounded design may be risk-routed rather than globally expensive, but it must not allow a final release gate to proceed without terminal CI + CodeQL on the actual final
mainSHA.This supplements #550's existing PR-run cancellation/sharding work; it is specifically about post-merge main evidence retention, not permission to weaken exact-head PR convergence.
qnbs commentedon Sep 20, 2026
2026-09-20 efficiency evidence — reviewer remediation can multiply full CI even when every push correctly waits
PR #779 provides a concrete pipeline-efficiency case for this issue.
Live timeline reconstruction showed that the autonomous agent did wait for the prior CI/CD wave to finish before each subsequent push. The waste did not come from overlapping CI. It came from a missing review-convergence stop condition:
The PR reached 14 source commits before squash merge. Several successive full CI cycles were roughly 20–30 minutes each, while reviewer/model quota was also consumed repeatedly.
#780 now owns the autonomous stop/disposition policy. #550 should incorporate the measurable efficiency side:
Do not “optimize” this by disabling useful reviewers or required gates. The target is bounded convergence and elimination of redundant revalidation, not weaker evidence.
qnbs commentedon Sep 23, 2026
Current-main build-hook cost sample — 2026-09-23
Resulting-main CI/CD #3463, Build job
107155146681, emitted Vite/Rolldown plugin timing evidence on exact currentmain2de980db...:The same Codecov hook then retried
get-pre-signed-urlthree times and ended in the known #607404 - Not Foundupload failure while the build and local bundle-analysis artifact still succeeded.This is a useful #550 optimization datum:
Record this sample in the whole-pipeline cost inventory; re-measure before implementing an optimization because plugin timings vary by runner/cache/head.
qnbs commentedon Sep 28, 2026
Mutation-evidence freshness handoff from #764
DEV.to follow-up on the mutation article highlighted that a manual-only mutation lane can leave an undated red/green status attached to code many commits newer than the last run.
#764 now owns the semantics: exact SHA/timestamp/run ID, explicit CURRENT/STALE/INVALID/NO_QUALIFIED evidence states, and no undated current-status claims.
This issue (#550) continues to own the cadence/cost decision only: whether a scheduled mutation run is justified, at what frequency/scope, and with what runner budget once #764 has trustworthy runner/scope evidence. Do not add schedule pressure before the lane is valid.
test(core): tighten the Gate 3A fault tests and scope the 3-OS step (#…
feat(core): add Gate 3 slice 3A durable staging and promotion (#445) (#…
feat(core): add Gate 3 slice 3A durable staging and promotion (#445) (#…
qnbs commentedon Oct 11, 2026
WorldScript control-plane reconciliation — 2026-10-11
Protected main is
c87d1cda171ac511ec65350ec7feed1780e829af. Push-triggered CI/CD run 38071809656 and CodeQL run 38071809617 are SUCCESS on that exact SHA. Latest published release remains v1.29.1.This is #550; do not import the colliding CannaGuide issue. #1017 is merged. A successful current-main run is not a measured efficiency/cost baseline. Before changing CI, record duplicate work and before/after time/cost while retaining Node, CodeQL, security, signing and governance checks. No cost inventory, workflow change or efficiency closure is claimed.
Gate 4 remains open. Gate 4E E1 is terminal; E2/E3 are not admitted. Gates 5/6/7 and v1.30 are not admitted. RELEASE_ALLOWED=NO; PRODUCTION_AUTHORITY_SWITCH_ALLOWED=NO.
Canonical mirror: https://linear.app/qnbs/issue/QNB-41. This publishes the revalidated QNB-194 handover packet through the existing maintainer gh principal. Historical specifications and issue state remain preserved; no source or workflow change is implied.
qnbs commentedon Oct 12, 2026
2026-10-12 — GitHub Pages pruning efficiency/safety split (cross-link; no CI gate relaxation)
A dedicated GitHub deployment-record maintenance owner now exists: #1028 —
ci(pages): restore safe continuous GitHub Pages deployment-record pruning. Desired steady state is newest 3github-pagesrecords plus any positively protected active/rollback/evidence records. This is not #627 Vercel Preview housekeeping. The former #522 Pages-deploy skip is closed; do not reopen without a reproduced new regression.Measured/source-bound starting points:
prune-deployments.ymlcurrently triggers on everyCI / CDcompletion, weekly cron, manual dispatch; groups all environments, keeps 3 per env, and continues toward deletion even ifcreateDeploymentStatus(inactive)fails.ci.yml's Pages deploy is scoped to successfulmainCI. PR CI 38144396221 was SUCCESS with Pages deploy correctly skipped. Do not count successful checks as savings evidence.Under #1028 Codex CLI may implement only directly coupled cleanup efficiency improvements: restrict eligible workflow_run events to actual successful
mainpush CI, bounded serialized writes, strict Pages ownership, safe dry-run and testable failure path. Collect before/afterworkflow_runtrigger counts and runner-minute estimates if observable; do not invent a metric or silence any existing required CI. Production and Vercel are different planes.#550 remains canonical for pipeline-wide cost/latency optimization; #510/#506/#875 own privileged governance, #723 owns incident/privacy evidence. The currently disabled pruning workflow must not be re-enabled without a separate explicit owner approval and post-merge dry-run/safety checks. A separate Linear issue is presently unavailable because of a workspace issue-cap error; until that is resolved, #1028 is the specialist source issue and QNB-41 holds its cross-link, not its implementation scope.
No CI code changed in this coordination note.
qnbs commentedon Oct 13, 2026
2026-10-13 — Cloud-first tool/CI cost triage (not new implementation admission)
Reviewed candidate inventory / release-stage decisions. Existing Core Rust and Tauri gates already run fmt/check/Clippy/tests on GitHub-hosted workers, with OS secure-store matrix, MSRV, OSV, JS/TS CodeQL, frontend tests, and manual Stryker. After Ubuntu workstation overload from rust-analyzer, local heavy analyses and editor language servers are intentionally out of scope. Preserve CI-first execution.
Now: do NOT create another runner, generic CI/meta PR, mandatory check or npm/Cargo duplication; minimize local build load, keep
ci:prepushlight; review measured exact-head CI cost only while working on a necessary release slice. Post-v1.30 measurement candidates: controlledcargo-nextestA/B againstcargo test, targetedcargo-llvm-covRust test-gap evidence, Rust CodeQL runtime/cost after a narrowly admitted security change, conditionalcargo-denylicense/source value andzizmorunique workflow findings. Measure wall time, runner minutes, misses, false positives, existing control overlap; stop if no distinct value. No extra CI churn before #1027→Gates4-7→tag gate; QNB-41 owns cost, NOT GH #1028's separately gated Pages write-path activation.qnbs commentedon Oct 13, 2026
Cost measurement consumer (2026-10-13): planned bounded Claude self-verification #1034, full execution plan https://linear.app/qnbs/document/worldscript-studio-claude-self-verification-umsetzungsplan-1034-qnb-bcced70eb0be. Compare per-PR extra model tokens, wall time, local disk/cache, GitHub-runner minutes, review churn, unique validated true findings and false positives against existing baseline, separated into known-issue replay vs previously unseen pilot. Avoid always-on heavyweight tests/Stop-hooks and duplicate reviewer retriggers; preserve all existing required deterministic quality/security gates. No CI changes admitted here.
qnbs commentedon Oct 13, 2026
Cost/quality measurement refinement (2026-10-13) for #1034. Measure before-first-push true/falsifiable findings; first external-review P1/P2 root-cause classes; semantic correction waves; duplicate and false-positive rates; real per-slice vs stacked-composite PR sizes; time-to-reproducer; tokens, wall-clock, local disk, GitHub runner minutes. Pilot >=5 known replays and >=2 subsequent unseen admitted PRs; separate retrospective and blind performance. Do not use the 75% anecdote as a target, add always-on hooks/heavy local Cargo, or reduce required security CI. Linear v2 plan.