Skip to content

intent: eval-seeding-deterministic - #444

Merged
yihanzhu merged 3 commits into
mainfrom
ystack/intent/eval-seeding-deterministic
Sep 27, 2026
Merged

yihanzhu merged 3 commits into
mainfrom
ystack/intent/eval-seeding-deterministic

Conversation

@yihanzhu

Copy link
Copy Markdown
Owner

Tracks #439

Adds the G1 intent for the deterministic part of step-8 child concern 10, eval seeding. Two catalog families are still declared with no seeds: malicious-instructions and reviewer-severity-false-positive-negative. The intent asks to seed both from real work without any model: malicious-instructions with a deterministic grader over at least 3 trials, one planted surface each, using R6.2's invariant-then-recompute rule (never a diff against the control); reviewer severity as a human-graded set of at least 5 trials from real recorded review verdicts, operator-labelled, passing only with zero missed Important findings. Risk high. Grader kinds and trial policies stay as the catalog lists them; model graders and any grader, trial-policy or threshold change are reserved; the operator labels the ground truth. Nothing enables before step 7 (#426) closes. Landing follows child concerns 5 and 9 per R8.

Intake acceptance (from the #439 acceptance record, RC-1; re-verified via the API before opening):

  • title_sha256: b9dfe60431f4d978c3ea5155147ca89e38cc782e8dcc9755da4f6ee793a305f8
  • body_sha256: ce96d36603f40dd58f5520e65d71db37909e623d4f8d14ccca1e060bf5d5c64c

review_size:standard

🤖 Generated with Claude Code

Tracks #439.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
@cloudflare-workers-and-pages

cloudflare-workers-and-pages Bot commented Sep 27, 2026 •

Copy link
Copy Markdown

Deploying ystack with  Cloudflare Pages  Cloudflare Pages

Latest commit: 533fa76
Status: ✅  Deploy successful!
Preview URL: https://d64a91c7.fabrica-6yx.pages.dev
Branch Preview URL: https://ystack-intent-eval-seeding-d.fabrica-6yx.pages.dev

View logs

@yihanzhu

Copy link
Copy Markdown
Owner Author

Codex reviewer (cross-vendor, read-only)

Reviewed-head: 1cff538
Reviewed-base: 9004630
reviewer: gpt-6-astra @ high

Posted verbatim by codex-review.sh (codex exec review --json --base refs/codex-review/444-32545/base in an isolated temp worktree, sandbox forced read-only). Comments only — Codex never pushes, approves, or merges.

The change adds only an intent document. Its proposed outcomes and safety constraints match the accepted step-8 requirements and existing eval catalog; no actionable defects were found. The diff whitespace check passed.

@yihanzhu

Copy link
Copy Markdown
Owner Author

Codex reviewer (cross-vendor, read-only)

Reviewed-head: 1a127df
Reviewed-base: a0ed10d
reviewer: gpt-6-astra @ high

Posted verbatim by codex-review.sh (codex exec review --json --base refs/codex-review/444-35381/base in an isolated temp worktree, sandbox forced read-only). Comments only — Codex never pushes, approves, or merges.

The intent-only change matches the accepted step-8 eval requirements and preserves the grading, authorization, and dependency boundaries. No actionable defects were found; executable code is unchanged and the diff passes whitespace checks.

@yihanzhu

Copy link
Copy Markdown
Owner Author

Codex reviewer (cross-vendor, read-only)

Reviewed-head: 533fa76
Reviewed-base: 29e91d4
reviewer: gpt-6-astra @ high

Posted verbatim by codex-review.sh (codex exec review --json --base refs/codex-review/444-39023/base in an isolated temp worktree, sandbox forced read-only). Comments only — Codex never pushes, approves, or merges.

The intent-only change matches the accepted step-8 eval requirements and preserves the existing safety boundaries. No actionable defects were found; executable behavior is unchanged, and git diff --check passed.

@yihanzhu
yihanzhu merged commit 272ec0f into main Sep 27, 2026
4 checks passed
@yihanzhu
yihanzhu deleted the ystack/intent/eval-seeding-deterministic branch September 27, 2026 19:08
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant