Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
76 changes: 76 additions & 0 deletions skills/secops/log-analysis/SKILL.md
Original file line number Diff line number Diff line change
Expand Up @@ -317,6 +317,55 @@ Step 5: Build timeline
-> Identify gaps in visibility (log sources not available)
```

### Step 8: Timeline Integrity and Evidence Handling

Before drawing conclusions from correlated logs, validate that the timeline is
internally consistent and that evidence can be traced back to preserved source
records. Do not treat a correlation as high confidence if timestamps,
collection windows, or log integrity are ambiguous.

**Timeline validation checks:**

```
LOG-TIME-01: Source timezone, UTC offset, or daylight-saving context is missing
LOG-TIME-02: Events from different sources are correlated without clock-skew tolerance
LOG-TIME-03: Ingestion timestamp is confused with event timestamp
LOG-TIME-04: Log gaps or dropped-event counters overlap the incident window
LOG-TIME-05: Parser normalization loses original event ID, raw timestamp, host, user, or source IP
LOG-TIME-06: Duplicate events are counted as separate actions without deduplication evidence
```

**Evidence integrity checks:**

```
LOG-EVID-01: Raw log source or export path is not recorded
LOG-EVID-02: Query text, filters, and time bounds are missing from the finding
LOG-EVID-03: Exported evidence lacks hash, immutable storage reference, or chain-of-custody note
LOG-EVID-04: Sensitive values are copied into the report instead of being redacted
LOG-EVID-05: Analyst conclusion relies on transformed fields without raw-event spot checks
```

**Required timeline metadata:**

| Metadata | Why it matters |
|---|---|
| Event timestamp | Time generated by the source system |
| Ingestion timestamp | Time received by SIEM or log pipeline |
| Source timezone / offset | Needed to normalize cross-region evidence to UTC |
| Clock-skew assumption | Prevents false ordering when sources are not synchronized |
| Raw event reference | Allows the finding to be revalidated from original evidence |
| Parser / normalization version | Explains field mappings and prevents silent schema drift |

**Confidence calibration:**

| Condition | Confidence impact |
|---|---|
| Raw events, UTC-normalized timestamps, and query text retained | Supports High confidence |
| Minor clock skew documented and bounded | Supports Medium to High confidence |
| Only transformed fields available, no raw spot check | Reduce at least one confidence level |
| Log source gap overlaps key event window | Treat as visibility gap; avoid definitive negative claims |
| Missing timezone or ingestion/event timestamp ambiguity | Do not assert exact sequence ordering |

---

## 4. Findings Classification
Expand Down Expand Up @@ -348,9 +397,11 @@ Produce log analysis findings in this structure:
| Field | Value |
|-------|-------|
| Time Window | [Start -- End, UTC] |
| Time Normalization | [Source timezones, UTC conversion, clock-skew tolerance] |
| Systems | [Hostnames, IPs, or network segments] |
| Users | [Usernames or "all users"] |
| Log Sources | [List of log sources analyzed] |
| Evidence Handling | [Raw source path/export ID, query text retained, hash/immutability reference] |

### Findings Summary
| # | Finding | Severity | ATT&CK Technique | Log Source | Evidence |
Expand All @@ -374,6 +425,15 @@ Produce log analysis findings in this structure:
|-----------------|--------|-------|-------------------|------------|
| [HH:MM:SS] | [Source] | [Description] | [T-ID] | [Suspicious / Benign / Confirmed malicious] |

### Timeline Integrity
| Check | Result | Evidence |
|-------|--------|----------|
| Timezone normalized | [Yes/No] | [source offsets and conversion notes] |
| Clock skew bounded | [Yes/No/N/A] | [NTP status or tolerance used] |
| Log gaps reviewed | [Yes/No] | [pipeline health or dropped event counters] |
| Raw-event spot check | [Yes/No] | [sample event IDs or export references] |
| Evidence preserved | [Yes/No] | [hash, immutable storage, ticket, or case ID] |

### Baseline Observations
[Any baseline deviations noted, with comparison to established norms]

Expand Down Expand Up @@ -451,6 +511,21 @@ A single Event ID can have very different meanings depending on the context. Eve

Attempting to identify anomalous behavior without knowing what normal behavior looks like leads to both false positives (flagging normal activity as suspicious) and false negatives (missing truly anomalous activity that blends into an unfamiliar baseline). Invest in baseline establishment for high-value log sources before relying on anomaly-based analysis.

### Pitfall 6: Building an Exact Timeline From Ambiguous Timestamps

Different sources may report local time, UTC, ingestion time, or device time.
If timezone, daylight-saving offset, and clock skew are not documented, an
apparently precise sequence can be wrong. Normalize to UTC, retain original
timestamps, and record the tolerance used for correlation windows.

### Pitfall 7: Reporting Transformed SIEM Fields Without Raw Evidence

SIEM parsers can rename fields, drop values, or merge duplicated events. A
finding based only on normalized fields should include at least one raw-event
spot check and the query text used to produce the result. Preserve exported
evidence with a hash or immutable case reference when the analysis may support
incident response, legal review, or customer notification.

---

## 8. Prompt Injection Safety Notice
Expand Down Expand Up @@ -478,3 +553,4 @@ This skill processes user-supplied content that may include raw log data, event
9. **AWS CloudTrail Event Reference** -- https://docs.aws.amazon.com/awscloudtrail/latest/userguide/cloudtrail-event-reference.html
10. **Azure Activity Log Schema** -- https://learn.microsoft.com/en-us/azure/azure-monitor/essentials/activity-log-schema
11. **NIST SP 800-61 Rev 2 -- Incident Handling Guide** -- https://csrc.nist.gov/publications/detail/sp/800-61/rev-2/final
12. **NIST SP 800-86 -- Guide to Integrating Forensic Techniques into Incident Response** -- https://csrc.nist.gov/publications/detail/sp/800-86/final
79 changes: 79 additions & 0 deletions skills/secops/log-analysis/tests/timeline-evidence-edge-cases.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,79 @@
# Log Analysis Timeline Evidence Edge Cases

These fixtures validate that log-analysis reports preserve timeline integrity
and avoid overconfident conclusions when timestamps, pipeline health, or raw
evidence are incomplete.

## Edge Case 1: Mixed Local Time and UTC

Input evidence:

```json
[
{
"source": "windows_security",
"event_id": 4624,
"event_time": "2026-06-06 01:58:03",
"timezone": "America/Los_Angeles",
"host": "ws-17",
"user": "alice"
},
{
"source": "azure_ad_signin",
"event_time": "2026-06-06T09:59:10Z",
"ingestion_time": "2026-06-06T10:04:33Z",
"user": "alice"
}
]
```

Expected output:

- Finding ID: `LOG-TIME-01` if timezone is missing for any source
- Timeline shows UTC-normalized timestamps and retains original timestamps
- Do not assert exact sequence ordering unless clock-skew tolerance is documented

## Edge Case 2: Log Pipeline Gap During Incident Window

Input evidence:

```yaml
incident_window_utc: "2026-06-06T02:00:00Z/2026-06-06T03:00:00Z"
pipeline_health:
source: edr_telemetry
dropped_events: 12943
outage_window_utc: "2026-06-06T02:15:00Z/2026-06-06T02:41:00Z"
finding_claim: "No process execution occurred after suspicious logon"
```

Expected output:

- Finding ID: `LOG-TIME-04`
- Visibility Gaps records the EDR outage
- Report avoids definitive negative claims about missing process execution
- Recommendations include pipeline recovery and alternate-source review

## Edge Case 3: Normalized Fields Without Raw Event Spot Check

Input evidence:

```yaml
siem_result:
query_id: q-8842
parser_version: auth_parser_v9
normalized_fields:
action: login_success
src_ip: 203.0.113.50
user: svc-build
timestamp_utc: "2026-06-06T04:12:00Z"
raw_export:
path: null
sha256: null
```

Expected output:

- Finding ID: `LOG-EVID-01`, `LOG-EVID-03`, or `LOG-EVID-05`

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Require all missing-evidence gates in the fixture

This fixture has no raw export path, no hash, and only normalized SIEM fields, so it simultaneously satisfies LOG-EVID-01, LOG-EVID-03, and LOG-EVID-05 as defined in the new skill checks. Using or lets a report pass while ignoring two of the missing-evidence conditions, which weakens the edge-case coverage for the evidence gates this test is meant to validate.

Useful? React with 👍 / 👎.

- Confidence is reduced because no raw-event spot check is available
- Evidence Handling records missing export path and hash
- Remediation requests raw export preservation and query text retention