Skip to content

Add PIR remediation verification gates#1454

Open
jddark62 wants to merge 1 commit into
UnitOneAI:mainfrom
jddark62:improve/post-incident-remediation-verification
Open

Add PIR remediation verification gates#1454
jddark62 wants to merge 1 commit into
UnitOneAI:mainfrom
jddark62:improve/post-incident-remediation-verification

Conversation

@jddark62

@jddark62 jddark62 commented Jun 6, 2026

Copy link
Copy Markdown

Skill Improvement ($50-150 Bounty)

Skill Modified

Skill name: post-incident-review
Skill path: skills/incident-response/post-incident-review/

What Was Wrong

Fixes #1398.

The skill produced a remediation plan with owners, priorities, deadlines, and tickets, but it did not require proof that a remediation actually worked before closure. This can create a false positive PIR outcome where a closed ticket is treated as root-cause remediation even when no retest, detection validation, control evidence, recurrence monitoring, or overdue escalation exists.

What This PR Fixes

  • Adds acceptance criteria and required evidence fields to remediation action tracking.
  • Adds a new remediation verification and closure-gates step.
  • Defines required verification evidence for preventive, detective, corrective/recovery, process/playbook, and recurrence-monitoring remediations.
  • Separates Implemented-Unverified, Verified, Monitoring, Overdue, and Risk Accepted states.
  • Adds report sections for remediation verification, independent verifier, evidence reviewed, retest/control validation, detection validation, recurrence monitoring, and residual-risk exceptions.
  • Adds follow-up schedule fields for verification review, recurrence watch period, and overdue escalation path.
  • Adds pitfalls for closing tickets without evidence and allowing overdue remediation to drift.

Evidence

Before (false positive PIR outcome):

pir:
  root_cause: internet-facing admin interface lacked MFA
  remediation:
    ticket: SEC-1842
    status: closed
    owner: identity-team
    action: enable MFA
  verification:
    retest: missing
    config_evidence: missing
    detection_rule_validation: missing
    recurrence_monitoring: missing

The old template could treat this as completed remediation because the ticket was closed.

After (now correctly handled):
The remediation plan requires acceptance criteria and evidence before closure, and the PIR output distinguishes Implemented-Unverified from Verified. Detective changes require test events, alert routing, and owner confirmation; recurrence monitoring and overdue risk treatment are explicit.

Test Cases Added/Updated

  • Added vulnerable test cases (tests/vulnerable/)
  • Added benign test cases (tests/benign/)
  • Existing content sanity checks pass

Validation performed:

  • python3 content sanity check for version, verification gate, closure states, verification report fields, and new pitfalls
  • git diff --check
  • Privacy scan for local paths/personal-name fragments

Bounty Tier

  • Minor ($50) — Doc update, small logic tweak, typo fix
  • Moderate ($100) — New edge case coverage, FP reduction with evidence
  • Substantial ($150) — Rewritten detection logic, major coverage expansion

Bounty Info

  • I have read and agree to the CONTRIBUTING.md bounty terms
  • Preferred payment method: To be provided privately if awarded

Bounty Info

  • I have read and agree to the CONTRIBUTING.md bounty terms.
  • Preferred payment method can be coordinated privately after maintainer acceptance.

/claim #1398

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[REVIEW] post-incident-review: add remediation verification gates

1 participant