Skip to content

tooling(reviews): build exact-head multi-reviewer orchestrator with Kimi fallback #931

Open

Description

@qnbs

Current PR evidence update — 2026-10-09

On PR #1017 ca26d16, Codex has completed with a new P1, volatile manual cursor invalidation across restart, while three earlier threads from the previous head are resolved. A future exact-head review aggregator must display this as FINDINGS, not CLEAN because the review completed. Keep it read-only/advisory and do not install an orchestrator or gate into C2c-2. Prioritize actual material finding resolution within #359 before adding another reviewer.


Live admission limit — 2026-10-09

For #1017, only read-only exact-head evidence collection/schema, not a new orchestrator/workflow or Kimi deployment, is admissible while R-15 C2c-2 is converging. The original Codex P1/P1/P2 threads on 44ba1be are resolved, but that does not constitute a clean Codex review for the newer ca26d16 HEAD. Quota and missing-signal states must remain explicit, without treating CodeRabbit/Sourcery silence as a pass.

Acceptance for future implementation: a single snapshot that binds provider, review body/thread/status, final SHA and blocking role; guards against stale/duplicate comments; reports all unresolved material findings without altering branch protection or claiming automatic approval; measures extra minutes/token/runner cost with #550. A model should not be allowed to create new merges, new source commits or privileged GitHub workflows in the review reader. Do not make this backlog tool mandatory for #1017/v1.30.


Parent / role

Child implementation owner of #780 for a repository-owned, provider-neutral review evidence orchestrator.

Linear planning mirror: QNB-122.

This is not a new merge authority. It normalizes existing advisory/deterministic evidence so agents and maintainers can reason about the exact current PR head without treating quota, silence or stale statuses as clean.

Goal

Produce one exact-head evidence view across, as available:

Codex
Cubic
CodeRabbit
DeepSource
CodeScene
CodeAnt
Sourcery
GitHub Actions required/deterministic gates

Kimi remains an optional independent redundancy/fallback plane, not the architectural center.

Phase 0 — may be pulled forward during v1.30 without preempting R-15

Define/test the provider-neutral schema and read-only aggregation contract using current PRs such as #930 as fixtures/evidence.

Required normalized states:

CLEAN
FINDINGS
PENDING
UNAVAILABLE
QUOTA_LIMITED
CONFIG_INCIDENT
STALE_HEAD
UNKNOWN_EXTERNAL

For every observation bind:

  • provider;
  • exact PR/head SHA;
  • signal kind (review body / inline thread / commit status / workflow);
  • blocking role;
  • timestamp;
  • finding IDs/severity where available;
  • unresolved/resolved disposition;
  • config authority where known.

Never infer CLEAN from silence, quota exhaustion or an old head.

Security/trust constraints

  • read-only collection by default;
  • no auto-merge;
  • no branch-protection/ruleset mutation;
  • no reviewer promotion to required authority;
  • no PR-controlled code executed with privileged secrets;
  • no pull_request_target path that executes untrusted PR code/dependencies;
  • no provider token exposed to PR-controlled code;
  • no automatic source mutation in the initial rollout;
  • exact immutable pins for any workflow/action/tool introduced;
  • output is evidence/advice; repository deterministic gates retain merge authority.

Correction-wave support

The orchestrator may help answer:

Which reviewers have completed on THIS head?
Which have material unresolved findings?
Which are quota-limited/unavailable?
Which statuses are stale/config incidents?
Is a correction wave already justified?
Did the new head invalidate prior clean evidence?

It must not auto-dismiss findings or decide that a material P1/P2 is invalid.

Adaptive-learning output

For validated recurring findings, emit a bounded learning candidate:

ROOT_CAUSE_CLASS
RECURRENCE_PATTERN
BEST_ENFORCEMENT_LAYER
EXISTING_OWNER
NEW_OWNER_REQUIRED?

Preferred enforcement:
type/API > test > deterministic guard > contract > agent instruction > prose.

Do not auto-edit AGENTS/policy files.

Kimi redundancy

Preserve QNB-122's Kimi concept as an optional additional reviewer when existing providers are unavailable or diversity is useful:

  • manual/read-only first;
  • repository-owned workflow if later admitted;
  • no unaudited mutable community action as trust anchor;
  • no write token to model process;
  • bounded/de-duplicated exact-head review;
  • advisory only.

Relationship to other owners

Acceptance

  • provider-neutral evidence schema defined;
  • exact-head/stale-head distinction proven;
  • quota/unavailable/config-incident states represented explicitly;
  • advisory vs deterministic blocking role represented without guessing live enforcement;
  • current reviewers can be aggregated read-only;
  • one current PR replay demonstrates correct convergence state;
  • no privileged mutation path introduced;
  • Kimi optional redundancy remains bounded and advisory;
  • output can feed recurrence-prevention triage without auto-editing policy;
  • costs/retrigger behavior measured with ci(efficiency): reduce redundant cost and latency without weakening required quality/security gates #550 before broad automation.

Scheduling

Phase 0 design/evidence can run as a non-preemptive support lane now. Any workflow installation, new secret/provider integration or governance mutation must remain a separate bounded reviewed step and must not derail the active v1.30/R-15 critical path.

Activity

  1. qnbs commented on Oct 11, 2026

    @qnbs
    OwnerAuthor

    WorldScript control-plane reconciliation — 2026-10-11

    Protected main is c87d1cda171ac511ec65350ec7feed1780e829af. Push-triggered CI/CD run 38071809656 and CodeQL run 38071809617 are SUCCESS on that exact SHA. Latest published release remains v1.29.1.

    An empty cursor[bot] review is not owner evidence or an independent Codex review. #1021 now has a Codex summary reporting completed review on c06f6cf; this is distinct from a submitted approving review and does not clear failing gates. #1017 is merged. No orchestrator, Kimi workflow, new mandatory reviewer, auto-merge policy or implementation slice is admitted.

    Gate 4 remains open. Gate 4E E1 is terminal; E2/E3 are not admitted. Gates 5/6/7 and v1.30 are not admitted. RELEASE_ALLOWED=NO; PRODUCTION_AUTHORITY_SWITCH_ALLOWED=NO.

    Canonical mirror: https://linear.app/qnbs/issue/QNB-122. This publishes the revalidated QNB-194 handover packet through the existing maintainer gh principal. Historical specifications and issue state remain preserved; no source or workflow change is implied.

  2. qnbs commented on Oct 13, 2026

    @qnbs
    OwnerAuthor

    Planned upstream read-only verifier (2026-10-13): #1034; detailed phase plan https://linear.app/qnbs/document/worldscript-studio-claude-self-verification-umsetzungsplan-1034-qnb-bcced70eb0be. The new Claude project agent/skill is an author-side, on-demand adversarial invariant counterexample finder, not this issue's provider-neutral exact-head review aggregator. If both mature, an optional, separately admitted read-only evidence adapter may expose HEAD, findings, tests and UNPROVEN to #931. No deployment, new workflow, token grants, automated review comments, enforced status, or parallel toolchain PR in #931 is authorized. Retain provider QUOTA_LIMITED/UNKNOWN distinct from CLEAN.

  3. qnbs commented on Oct 13, 2026

    @qnbs
    OwnerAuthor

    Exact-head evidence interface from #1034 (2026-10-13). Normalize three scopes: current PR-base to HEAD cumulative CI/governance diff; previous admitted slice HEAD to current HEAD independent semantic delta; last substantive review HEAD to current HEAD invalidation delta. Essential for #1033/#1035 (the child composite diff is NOT independent E2c). If separately admitted later, ingest only read-only self-verifier hypotheses with SHA and an external deterministic oracle; do not equate them with independent Codex, normalize quota/no-signal as clean, or install a new required check/privileged workflow. Linear v2 plan.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions