[REVIEW] hipaa-review: add audit control coverage and integrity evidence gates
Skill Being Reviewed
Skill name: hipaa-review
Skill path: skills/compliance/hipaa-review/
False Positive Analysis
Benign-looking HIPAA audit controls result that can be incorrectly scored as compliant:
164.312(b) Audit Controls: Compliant
Evidence:
- SIEM dashboard screenshot
- EHR logging enabled
- Logs retained 90 days
- Security team reviews alerts weekly
Why this is a false positive:
The skill currently asks whether logs are enabled, reviewed, and retained, but does not require proof that every ePHI system is mapped to a log source, that patient-record access and export events are captured, that logs cannot be altered by ordinary administrators, or that reviews are documented with sampled systems and follow-up. A generic SIEM screenshot can hide missing EHR audit trails, unlogged database exports, Business Associate blind spots, or mutable local logs.
Coverage Gaps
Missed variant 1: EHR audit enabled but database exports are unlogged
EHR user access logs: enabled
Reporting database exports: no per-user export log
Bulk CSV downloads: service account only
Review result: compliant
Why it should be caught: HIPAA audit controls should cover systems that create, receive, maintain, or transmit ePHI, not only the primary EHR UI.
Missed variant 2: Logs exist but are mutable by application administrators
Audit destination: local application table
App admins: can update/delete audit rows
SIEM forwarding: disabled for cost reasons
Why it should be caught: Audit records that can be altered by ordinary admins do not provide reliable evidence for review, incident investigation, or OCR audit readiness.
Missed variant 3: Business Associate ePHI system lacks audit visibility
BA hosted portal: stores ePHI
BAA: executed
Audit log access: available only on request, no SLA
Last audit report: never requested
Why it should be caught: BA-managed ePHI systems still need audit visibility through log access, periodic reports, or contract evidence that audit records can be produced during investigations.
Edge Cases
- Legacy clinical systems produce authentication logs but no patient-record-level access events.
- Shared service accounts collapse identity fidelity for exports or API access.
- Cloud/SaaS logs are retained for less time than the organization's risk analysis and policies require.
- Review artifacts show meetings occurred but not which systems/users were sampled or how anomalies were closed.
Remediation Quality
Comparison to Other Tools
| Tool |
Catches this? |
Notes |
| HHS OCR HIPAA Audit Protocol |
Partial |
Includes Security Rule audit-control inquiry, but skill should operationalize evidence expectations |
| NIST SP 800-66 Rev. 2 |
Partial |
Provides implementation guidance for HIPAA Security Rule controls; skill needs concrete output fields |
| SIEM compliance dashboards |
Partial |
Can show received events, but may not prove ePHI-system coverage or audit-log integrity |
Overall Assessment
Strengths: Strong safeguard-by-safeguard HIPAA Security Rule structure, accurate required/addressable framing, and clear output roadmap.
Needs improvement: Audit Controls are currently too terse for real OCR audit readiness. They need concrete evidence fields that prevent generic logging claims from being scored as compliant.
Priority recommendations:
- Add an ePHI audit trail source coverage matrix.
- Require audit-log integrity protection and identity fidelity evidence.
- Add output fields for audit control status, retention, last review, and gaps.
Bounty Info
[REVIEW] hipaa-review: add audit control coverage and integrity evidence gates
Skill Being Reviewed
Skill name:
hipaa-reviewSkill path:
skills/compliance/hipaa-review/False Positive Analysis
Benign-looking HIPAA audit controls result that can be incorrectly scored as compliant:
Why this is a false positive:
The skill currently asks whether logs are enabled, reviewed, and retained, but does not require proof that every ePHI system is mapped to a log source, that patient-record access and export events are captured, that logs cannot be altered by ordinary administrators, or that reviews are documented with sampled systems and follow-up. A generic SIEM screenshot can hide missing EHR audit trails, unlogged database exports, Business Associate blind spots, or mutable local logs.
Coverage Gaps
Missed variant 1: EHR audit enabled but database exports are unlogged
Why it should be caught: HIPAA audit controls should cover systems that create, receive, maintain, or transmit ePHI, not only the primary EHR UI.
Missed variant 2: Logs exist but are mutable by application administrators
Why it should be caught: Audit records that can be altered by ordinary admins do not provide reliable evidence for review, incident investigation, or OCR audit readiness.
Missed variant 3: Business Associate ePHI system lacks audit visibility
Why it should be caught: BA-managed ePHI systems still need audit visibility through log access, periodic reports, or contract evidence that audit records can be produced during investigations.
Edge Cases
Remediation Quality
Comparison to Other Tools
Overall Assessment
Strengths: Strong safeguard-by-safeguard HIPAA Security Rule structure, accurate required/addressable framing, and clear output roadmap.
Needs improvement: Audit Controls are currently too terse for real OCR audit readiness. They need concrete evidence fields that prevent generic logging claims from being scored as compliant.
Priority recommendations:
Bounty Info