Skip to content

SUPPRESSION.md

Latest commit

 

History

History
119 lines (90 loc) · 5.29 KB

SUPPRESSION.md

File metadata and controls

119 lines (90 loc) · 5.29 KB

Baseline / False-Positive Suppression

SkillSpector's analyzers — especially the LLM semantic ones — can produce findings that are correct in general but not actionable for your skills (framework/architectural patterns, first-party tooling conventions, accepted lab practices). A baseline lets you suppress those known findings so that:

  • the risk score reflects only un-triaged issues,
  • re-scans surface only new findings (incremental CI/CD), and
  • every suppression carries an auditable reason.

Suppressed findings never count toward the risk score and are excluded from the SARIF results. They are shown in the terminal/Markdown report only when you pass --show-suppressed, and are always listed (machine-readable) in the JSON report under suppressed / suppressed_count.

Addresses issue #88.

Quick start

# 1. Accept all current findings into a baseline (run once).
skillspector baseline ./my-skill/ -o .skillspector-baseline.yaml

# 2. Commit the baseline, then scan against it. Only NEW findings are reported.
skillspector scan ./my-skill/ --baseline .skillspector-baseline.yaml

# Review what was suppressed.
skillspector scan ./my-skill/ --baseline .skillspector-baseline.yaml --show-suppressed

CLI

Command / option Description
skillspector baseline <path> [-o FILE] [--no-llm] [--reason TEXT] Scan and write a baseline that fingerprint-suppresses every current finding. Default output: .skillspector-baseline.yaml.
skillspector scan <path> --baseline FILE (-b) Suppress findings matching the baseline before scoring/reporting.
skillspector scan <path> --baseline FILE --show-suppressed Also list the suppressed findings (they still don't affect the score).

A missing or malformed baseline file exits with code 2.

Baseline file format

YAML or JSON (the .json extension selects JSON output when generating). Two complementary mechanisms:

version: 1

rules:                       # human-authored, glob-based, drift-tolerant
  - id: "SQP-1"              # glob over the finding's rule id
    reason: "Trigger-phrase breadth is a description nit, not a vuln"
  - id: "SSD-2"
    path: "example-skill/SKILL.md"       # glob over the finding's file
    message: "*example false-positive phrase*"   # glob over the finding's message
    reason: "False positive: benign trigger phrase, not an instruction"

fingerprints:                # machine-generated, exact
  - hash: "sha256:1a2b3c4d5e6f7081"
    rule_id: "SDI-2"         # informational (for humans reading the file)
    file: "example-skill/SKILL.md"
    reason: "Accepted — reads its own environment for context"

rules — glob suppression

A finding is suppressed when every field a rule specifies matches it; unspecified fields match anything. Use this for:

  • Global pattern suppression — id: "SQP-1" (or id: "SQP-*") drops a rule or rule family across all skills.
  • Skill/file-scoped suppression — add path: (and optionally message:) to scope the suppression to a specific skill, file, or message.

Field reference:

Field Matches against Notes
id (or rule_id) Finding.rule_id glob
path (or file) Finding.file glob; * crosses /, ** is an alias for *
message Finding.message glob, case-insensitive; wrap a keyword in * for substring
reason — required; recorded in reports and audits

Glob matching uses Python's fnmatch, so * matches across path separators (*SKILL.md matches a/b/SKILL.md). Rules are drift-tolerant: they keep working after line numbers shift or content is reworded.

fingerprints — exact suppression

Each entry is the stable hash of one finding (sha256(rule_id|file|start_line|end_line|message), truncated). Generated by skillspector baseline. Because the hash includes the line span and message, editing a skill so a finding moves or is reworded changes its fingerprint — regenerate the baseline after material changes, or prefer rules for suppressions you want to survive edits.

An entry may be a bare string ("sha256:...") or a mapping with hash, optional reason, and informational rule_id / file.

How it fits the pipeline

Suppression is applied in the report node (skillspector/nodes/report.py), the single place where findings are scored and formatted, so the CLI and any future REST API behave identically. The CLI loads the baseline file into a skillspector.suppression.Baseline and passes it via graph state (state["baseline"], state["show_suppressed"]); the report node partitions findings into kept vs. suppressed via skillspector.suppression.partition_findings.

Recommended workflow

  1. Triage the first scan. For genuine false positives, prefer a rules entry with a clear reason (drift-tolerant). For "accept everything as-is right now", run skillspector baseline to fingerprint them.
  2. Commit the baseline file to the repo.
  3. In CI, run skillspector scan <path> --baseline <file>; the build fails (exit 1) only when a new finding pushes the risk score above threshold.
  4. Periodically review with --show-suppressed and prune stale entries.