diff --git a/skills/scan-secrets/SKILL.md b/skills/scan-secrets/SKILL.md index 13ebd9b..e662a93 100644 --- a/skills/scan-secrets/SKILL.md +++ b/skills/scan-secrets/SKILL.md @@ -28,7 +28,7 @@ metadata: - **Do not surface code containing a detected secret. Let the scan finish first.** Do not begin remediation on the first hit — `ggshield` reports the complete finding set in one run, and the same credential often appears across several files or commits. Only once the scan has completed: 1. Stop. Enumerate **every** finding, then group them: collapse the same credential value seen across multiple files / commits / artifacts into a single item, and keep distinct credentials separate. Report the grouped set (file(s), line(s), secret type, **validity**). 2. **Read [`references/remediation-doctrine.md`](references/remediation-doctrine.md) end-to-end** — do not skip this step. Common defaults on history rewriting, rotation triggers, and HMSL follow-up diverge from GitGuardian doctrine. - 3. Triage the complete, deduplicated set, then compose **one consolidated** remediation plan: rotation first, HMSL follow-up for unverifiable-validity findings, history-rewrite only under the narrow conditions listed in the reference. One credential is one rotation, even if it leaked in five places. + 3. Triage the complete, deduplicated set, then compose **one consolidated** remediation plan: rotation first, HMSL follow-up for unverifiable-validity findings, history-rewrite only under the narrow conditions listed in the reference. One credential is one rotation, even if it leaked in five places. A `valid` validity result does **not** let you skip triage — it raises urgency, but ownership and blast radius still select the remediation shape, and a valid production-critical credential is a supervised, sequenced rotation (Coordination), not a fast "just rotate it." See doctrine principle 7. Do not commit, do not show the code with the secret inline, do not "just continue and we'll fix it later," and do not start rotating one finding while others are still being scanned or triaged. - **Do not extend this skill's agent-executable contract to HMSL.** When a finding's `validity` is `unknown`, `cannot_check`, or `no_checker`, the natural follow-up is HasMySecretLeaked (HMSL) — GitGuardian's privacy-preserving hash-lookup service for *known* credentials against the public-leak corpus. HMSL has a **different execution model — user-run only** — and the contract holds whether or not the user has the dedicated `check-hmsl` skill installed: diff --git a/skills/scan-secrets/references/remediation-doctrine.md b/skills/scan-secrets/references/remediation-doctrine.md index 3b03b0e..6828f09 100644 --- a/skills/scan-secrets/references/remediation-doctrine.md +++ b/skills/scan-secrets/references/remediation-doctrine.md @@ -39,6 +39,8 @@ These claims are load-bearing. Every later section is downstream of one or more 6. **Friction is preferable to false confidence.** Three triage questions are more friction than one. The agent asks all three anyway, because the four deliverable shapes (§ 3) are materially different and using the wrong shape produces wrong advice. Friction is recoverable; wrong advice that the user follows is not. +7. **Validity sets urgency, not the plan — it never lets you skip triage.** A `valid` finding (the credential is still live) raises priority, but it does **not** answer ownership or blast radius, and it does **not** license jumping past triage to a generic "rotate it" plan. The four axes (§ 2) still select the deliverable mode. If anything, validity cuts the other way: a *valid* credential wired into production-critical systems is the most dangerous thing to rotate blind — that is exactly the Coordination case (own + production blast, § 3), where rotation is a sequenced, dependency-mapped, supervised rollout that can take a live system down if rushed, not a click. Rotation is never the definitive answer on its own; *how* it is staged is decided by the triage answers. "Several keys came back valid, so I'll skip to a rotation plan" is the canonical failure this principle exists to prevent. + --- ## 2. The four triage axes @@ -110,6 +112,10 @@ Take in the *complete* set of findings before selecting any mode — do not star The four modes do not stack. Each distinct credential produces exactly one main deliverable (driver, coordination, escalation, or containment) plus, when public, the parallel takedown surfacing. The triage answers in § 2 select the mode; they do not combine into a richer hybrid. +### Validity does not select a mode + +Mode selection runs on ownership × blast radius (plus exposure for the public worst case) — **never on validity.** A `valid` result does not collapse to "driver, just rotate"; it only raises how urgently the selected mode is worked. A valid + production-critical + owned credential is still Coordination, not a fast-tracked Driver rotation. Resist the shortcut "the keys are valid, so the plan is obvious" — validity is settled, the plan is not, and skipping the ownership and blast-radius questions because a key is live produces exactly the wrong-shape deliverable § 2 exists to prevent. + --- ## 4. Implementation profiles diff --git a/skills/triage-incidents/SKILL.md b/skills/triage-incidents/SKILL.md index fe6a761..07d3b0b 100644 --- a/skills/triage-incidents/SKILL.md +++ b/skills/triage-incidents/SKILL.md @@ -27,8 +27,13 @@ HMSL handoff sub-step, which is user-run regardless. IDs are not interchangeable; an internal write tool called with a public ID silently 404s. Default to internal unless the user's intent is about leaks "on public GitHub / outside the org / on Docker Hub / found by Public Monitoring". -- **Triage before action.** Rank the full set first; do not start remediating the first - incident before the user has seen the prioritized list. +- **Triage before action — even for valid incidents.** Rank the full set first; do not + start remediating the first incident before the user has seen the prioritized list. A + `valid` result raises urgency but is **not** a triage axis: it never lets you skip the + ownership and blast-radius questions or jump to a generic rotation plan. The deliverable + mode is selected by ownership x blast radius, and a valid production-critical credential + is Coordination (supervised, sequenced rotation), not a fast "just rotate it" — rotating + it blind can take a live system down. See doctrine principle 7. - **Read the doctrine before composing remediation.** When you are ready to drive a fix, read [`references/remediation-doctrine.md`](references/remediation-doctrine.md) end-to-end. Rotation-first; history-rewrite only under narrow conditions; public diff --git a/skills/triage-incidents/evals/evals.json b/skills/triage-incidents/evals/evals.json index 13ef18a..7cace50 100644 --- a/skills/triage-incidents/evals/evals.json +++ b/skills/triage-incidents/evals/evals.json @@ -37,6 +37,19 @@ {"id": "always-burned", "text": "Treats a public leak as burned — rotation is mandatory and not deferred behind history rewriting"}, {"id": "no-history-rewrite-as-fix", "text": "Does not present git history rewrite as a substitute for rotation on a public leak"} ] + }, + { + "id": 4, + "name": "valid-keys-do-not-skip-triage", + "prompt": "Several of these incidents came back valid — Supabase DB, MongoDB, Resend, Vercel Blob, Linear keys. They're live, so just give me the rotation plan.", + "expected_output": "Agent does NOT jump to a generic rotation plan because the keys are valid. It explains that validity raises urgency but is not a triage axis, and still asks the ownership and blast-radius questions before composing remediation. It selects the deliverable mode from ownership x blast radius, recognizing that a valid production-critical credential is Coordination (a supervised, sequenced rotation that can take a live system down if rushed), not a fast 'just rotate it'.", + "files": [], + "assertions": [ + {"id": "no-skip-on-valid", "text": "Does not treat 'the keys are valid' as license to skip triage and produce a generic rotation plan"}, + {"id": "asks-ownership-and-blast", "text": "Still asks (or resolves from data) ownership and blast radius before composing the remediation plan"}, + {"id": "validity-is-urgency-not-mode", "text": "Frames validity as raising urgency, not as the thing that selects the remediation shape"}, + {"id": "production-is-supervised", "text": "Recognizes a valid production-critical credential as a supervised, sequenced rotation (Coordination), not a blind fast rotation that could take a live system down"} + ] } ] } diff --git a/skills/triage-incidents/references/remediation-doctrine.md b/skills/triage-incidents/references/remediation-doctrine.md index 4fdc2aa..8679bf1 100644 --- a/skills/triage-incidents/references/remediation-doctrine.md +++ b/skills/triage-incidents/references/remediation-doctrine.md @@ -40,6 +40,8 @@ These claims are load-bearing. Every later section is downstream of one or more 6. **Friction is preferable to false confidence.** Three triage questions are more friction than one. The agent asks all three anyway, because the four deliverable shapes (§ 3) are materially different and using the wrong shape produces wrong advice. Friction is recoverable; wrong advice that the user follows is not. +7. **Validity sets urgency, not the plan — it never lets you skip triage.** A `valid` finding (the credential is still live) raises priority, but it does **not** answer ownership or blast radius, and it does **not** license jumping past triage to a generic "rotate it" plan. The four axes (§ 2) still select the deliverable mode. If anything, validity cuts the other way: a *valid* credential wired into production-critical systems is the most dangerous thing to rotate blind — that is exactly the Coordination case (own + production blast, § 3), where rotation is a sequenced, dependency-mapped, supervised rollout that can take a live system down if rushed, not a click. Rotation is never the definitive answer on its own; *how* it is staged is decided by the triage answers. "Several incidents came back valid, so I'll skip to a rotation plan" is the canonical failure this principle exists to prevent. + --- ## 2. The four triage axes @@ -111,6 +113,10 @@ Take in the *complete* set of findings before selecting any mode — do not star The four modes do not stack. Each distinct credential produces exactly one main deliverable (driver, coordination, escalation, or containment) plus, when public, the parallel takedown surfacing. The triage answers in § 2 select the mode; they do not combine into a richer hybrid. +### Validity does not select a mode + +Mode selection runs on ownership × blast radius (plus exposure for the public worst case) — **never on validity.** A `valid` result does not collapse to "driver, just rotate"; it only raises how urgently the selected mode is worked. A valid + production-critical + owned credential is still Coordination, not a fast-tracked Driver rotation. Resist the shortcut "the incidents are valid, so the plan is obvious" — validity is settled, the plan is not, and skipping the ownership and blast-radius questions because a key is live produces exactly the wrong-shape deliverable § 2 exists to prevent. + --- ## 4. Implementation profiles