ci(evals): stop native evals triggering on trusted .github CI files

[mrizzi]

Summary

• Drop the .github/... alternative from nativeRelevant in the discover job of eval-pr-run.yml. The native run never executes PR-side copies of eval-pr.yml, eval-pr-run.yml (workflow_run uses main's copy), run-native-fullsend-evals.sh or check-native-fullsend-execution.py (run from the trusted base checkout).
• Flip the two .github/scripts/* discovery test rows to native == "false" and add rows for eval-pr-run.yml / eval-pr.yml.
• Trust check, gate/run-evals/run-native-evals conditions and eval-protected are unchanged.

Motivation: Dependabot #331 (touching only eval-pr-run.yml) waited ~3h40m on eval-protected and then ran ~18 min of native evals that tested nothing from the PR.

Test plan

• python3 -m pytest plugins/sdlc-workflow/scripts/ -q — 683 passed, 2 skipped
• uvx skillsaw — 0 errors
• claude plugin validate plugins/sdlc-workflow — passed

Implements TC-6912

🤖 Generated with Claude Code

Summary by Sourcery

Prevent unnecessary native evaluations when pull requests only modify trusted CI workflow or execution-helper files.

Bug Fixes:

• Prevent native evals from being triggered by changes to trusted .github workflow and native-evaluation helper files that are not executed from the pull request checkout.

Enhancements:

• Update native-evaluation discovery coverage to classify the affected .github files as non-native-relevant while preserving native detection for evaluated project files.

Tests:

• Update discovery tests to cover both evaluation workflows and native-evaluation helper scripts.

[sourcery-ai]

Reviewer's guide (collapsed on small PRs)

Reviewer's Guide

The PR prevents changes limited to trusted .github CI orchestration or native-eval execution files from triggering unnecessary native evaluations by removing those paths from nativeRelevant, and updates discovery fixtures to enforce the new behavior while leaving trust and downstream gating logic unchanged.

Flow diagram for native evaluation discovery gating

flowchart TD
    A[Changed files in pull request] --> B{nativeRelevant matches}
    B -->|Native eval inputs or implementation changed| C[native = true]
    B -->|Only trusted .github CI or native runner files changed| D[native = false]
    C --> E[Native evaluations run]
    D --> F[Native evaluations skipped]
    D --> G[eval-protected and downstream gating remain unchanged]

Loading

File-Level Changes

Change	Details	Files
Restrict native-eval relevance detection to PR-controlled evaluation inputs and plugin files, excluding trusted CI orchestration and execution scripts.	Remove the .github workflow and native-eval script patterns from nativeRelevant. Preserve native relevance for evaluation content and the listed SDLC workflow plugin artifacts.	.github/workflows/eval-pr-run.yml
Update discovery tests to codify that trusted CI files do not trigger native evaluations.	Change native expectations for the native-eval shell and Python scripts to false. Add coverage for eval-pr-run.yml and eval-pr.yml as non-native-triggering changes.	plugins/sdlc-workflow/scripts/test_native_fullsend_eval_ci.py

---

Tips and commands

Interacting with Sourcery

• Trigger a new review: Comment @sourcery-ai review on the pull request.
• Continue discussions: Reply directly to Sourcery's review comments.
• Generate a GitHub issue from a review comment: Ask Sourcery to create an
  issue from a review comment by replying to it. You can also reply to a
  review comment with @sourcery-ai issue to create an issue from it.
• Generate a pull request title: Write @sourcery-ai anywhere in the pull
  request title to generate a title at any time. You can also comment
  @sourcery-ai title on the pull request to (re-)generate the title at any time.
• Generate a pull request summary: Write @sourcery-ai summary anywhere in
  the pull request body to generate a PR summary at any time exactly where you
  want it. You can also comment @sourcery-ai summary on the pull request to
  (re-)generate the summary at any time.
• Generate reviewer's guide: Comment @sourcery-ai guide on the pull
  request to (re-)generate the reviewer's guide at any time.
• Resolve all Sourcery comments: Comment @sourcery-ai resolve on the
  pull request to resolve all Sourcery comments. Useful if you've already
  addressed all the comments and don't want to see them anymore.
• Dismiss all Sourcery reviews: Comment @sourcery-ai dismiss on the pull
  request to dismiss all existing Sourcery reviews. Especially useful if you
  want to start fresh with a new review - don't forget to comment
  @sourcery-ai review to trigger a new review!

Customizing Your Experience

Access your dashboard to:

• Enable or disable review features such as the Sourcery-generated pull request
  summary, the reviewer's guide, and others.
• Change the review language.
• Add, remove or edit custom review instructions.
• Adjust other review settings.

Getting Help

• Contact our support team for questions or feedback.
• Visit our documentation for detailed guides and information.
• Keep in touch with the Sourcery team by following us on X/Twitter, LinkedIn or GitHub.

[sourcery-ai]

Hey - I've reviewed your changes and they look great!

Sourcery assessment

Approved.

---

Sourcery is free for open source - if you like our reviews please consider sharing them ✨

• X
• Mastodon
• LinkedIn
• Facebook

[fullsend-ai-review]

🤖 Finished Verify Pr · ✅ Success · Started 10:49 AM UTC · Completed 10:58 AM UTC

Commit: 7b883e3 · View workflow run →

Runtime: claude · Model: claude-opus-4-8 · Effort: high · Cost: $4.26

[fullsend-ai-review]

Verification Report for TC-6912 (commit 7b883e3)

Check	Result	Details
Review Feedback	N/A	No inline comments or substantive review-body feedback (1 Sourcery approval, no code change requests).
Root-Cause Investigation	N/A	No sub-tasks created; nothing to investigate.
Scope Containment	PASS	PR files exactly match the task's Files to Modify; no out-of-scope or unimplemented files.
Diff Size	PASS	9 lines across 2 files - proportionate to a targeted CI-expression + test change.
Commit Traceability	PASS	The sole commit references TC-6912 (Implements TC-6912).
Sensitive Patterns	PASS	No secrets, credentials, or sensitive data in added lines.
CI Status	PASS	All 8 head-SHA checks succeeded, incl. Script Unit Tests on Python 3.11-3.14.
Acceptance Criteria	PASS	3 of 3 criteria met.
Test Quality	PASS	No repetitive tests; no undocumented new test functions; Eval Quality: N/A.
Test Change Classification	ADDITIVE	2 parametrized cases added + 2 expected values corrected; no coverage removed.
Verification Commands	PASS	pytest suite passed via CI (3.11-3.14); no eval-infra changes.

Overall: PASS

All checks passed or were N/A. The PR cleanly implements TC-6912: it removes the .github/... alternative from the nativeRelevant expression in the discover job of .github/workflows/eval-pr-run.yml, and updates the parametrized discovery test in test_native_fullsend_eval_ci.py so all four .github/ CI paths expect native == "false" (flipping the two .github/scripts/* rows and adding cases for eval-pr-run.yml and eval-pr.yml). Trust/approval gating and the plugin/eval positive cases are untouched. No review feedback required action; no sub-tasks or root-cause tasks were created. A human reviewer decides whether to merge.

---

This comment was AI-generated by sdlc-workflow/verify-pr v0.13.10.

[github-actions]

Native Fullsend Eval Results

Head: 7b883e3
Merge: f1e24be
Trusted workflow: 5436ac4
Reviewed eval source: 5436ac4

Native job: failure; complete: false.
Execution evidence: invalid; quality score (advisory): unavailable (grading incomplete; 21/28 judgments available, 21 passed / 0 failed).

• 037-release / assertion_1: scorer-error (phase-exit)
• 037-release / assertion_2: scorer-error (phase-exit)
• 037-release / assertion_3: scorer-error (phase-exit)
• 037-release / assertion_4: scorer-error (phase-exit)
• 037-release / assertion_5: scorer-error (phase-exit)
• 037-release / assertion_6: scorer-error (phase-exit)
• 037-release / assertion_7: scorer-error (phase-exit)

Case	Execution evidence	Quality score (advisory)
033-absent	valid	4/4
034-empty	valid	5/5
035-malformed	valid	5/5
036-valid	valid	7/7
037-release	valid	unavailable (0/7 judgments, 0 passed / 0 failed)