fix: validity never short-circuits triage in remediation doctrine#90
Merged
Conversation
A valid finding was being read as license to skip the ownership and
blast-radius triage and jump straight to a generic "rotate it" plan.
That inverts the doctrine: validity sets urgency, not the plan, and a
valid production-critical credential is the most dangerous thing to
rotate blind (Coordination — a supervised, sequenced rollout), not a
fast rotation.
- Add doctrine principle 7 ("Validity sets urgency, not the plan") and a
"Validity does not select a mode" note to both the scan-secrets and
triage-incidents remediation doctrines (kept structurally diffable).
- Guard the routing layer: triage-incidents SKILL.md "Triage before
action — even for valid incidents" and scan-secrets SKILL.md step 3.
- Add triage-incidents eval #4 locking in the valid-keys-do-not-skip-
triage behavior.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
An agent driving incident remediation skipped the ownership and blast-radius triage and jumped straight to a generic "rotate it" plan, rationalizing it as: several keys came back valid, so rotation is unconditional regardless of the answers.
That inverts the doctrine. The four triage axes are detection context, exposure, ownership, and blast radius — validity is not one of them. The doctrine said "triage first" and "rotation is necessary but not sufficient," but never named the specific trap: treating a
validresult as a substitute for triage. So nothing forbade the shortcut.Rotation is also not the definitive answer on its own. A valid credential wired into production-critical systems is the most dangerous thing to rotate blind — that is Coordination mode (a supervised, sequenced, dependency-mapped rollout that can take a live system down if rushed), not a fast "just rotate it."
Fix
Doctrine (
scan-secrets+triage-incidents, kept structurally diffable):Routing layer:
triage-incidentsSKILL.md: "Triage before action — even for valid incidents."scan-secretsSKILL.md step 3: avalidresult does not let you skip triage.Regression lock-in:
triage-incidentseval feat(skills): scaffold multi-skill layout and add honeytokens skill #4 (valid-keys-do-not-skip-triage) replays the scenario and asserts the agent still runs ownership + blast radius and treats a valid production-critical credential as supervised Coordination.Verification
npm test— 11/11 passevals.jsonvalidates🤖 Generated with Claude Code