Initial scaffold: structure, CI, starter content

Author: liliyke

.github/workflows/ci.yml

@@ -0,0 +1,26 @@
+name: CI
+
+on:
+  push:
+    branches: [main]
+  pull_request:
+
+jobs:
+  test:
+    runs-on: ubuntu-latest
+    steps:
+      - uses: actions/checkout@v4
+
+      - uses: actions/setup-python@v5
+        with:
+          python-version: "3.11"
+
+      - run: pip install uv
+
+      - run: uv pip install --system -e ".[dev]"
+
+      - name: Ruff
+        run: ruff check src/
+
+      - name: Tests
+        run: pytest -q

.gitignore

@@ -0,0 +1,21 @@
+__pycache__/
+*.py[cod]
+.venv/
+venv/
+.env
+.pytest_cache/
+.ruff_cache/
+.mypy_cache/
+*.egg-info/
+dist/
+build/
+
+.vscode/
+.idea/
+.DS_Store
+Thumbs.db
+
+# Generated outputs
+converted/
+*.jsonl
+navigator_layer.json

LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2026 Isaac Ojeh
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

README.md

@@ -0,0 +1,56 @@
+# detection-tooling
+
+Python utilities I use to operate detection content as code: convert Sigma rules between SIEM backends, score MITRE ATT&CK coverage, validate rule metadata, generate test telemetry, and normalize logs.
+
+These are intended as small, focused tools — each one does a single job well, reads its inputs from stdin or `--file`, and writes structured output suitable for piping into the next tool.
+
+## What's inside
+
+| Script | One-liner | Reads | Writes |
+|---|---|---|---|
+| [`sigma_to_sentinel`](src/detection_tooling/sigma_to_sentinel.md) | Convert a Sigma rule to a Sentinel analytic rule (KQL + ARM template) | Sigma YAML | ARM JSON |
+| [`attack_coverage`](src/detection_tooling/attack_coverage.md) | Build an ATT&CK Navigator layer from a folder of Sigma rules | Sigma YAML | Navigator JSON |
+| [`rule_validator`](src/detection_tooling/rule_validator.md) | Lint Sigma rules — schema, UUIDs, naming, required tags | Sigma YAML | Pass/fail report |
+| [`telemetry_generator`](src/detection_tooling/telemetry_generator.md) | Emit synthetic Sysmon-style events to test detections | Template YAML | JSONL events |
+| [`log_normalizer`](src/detection_tooling/log_normalizer.md) | Map raw Windows/Linux/JSON logs to ECS-style fields | Raw logs | ECS JSON |
+
+## Install
+
+```bash
+git clone https://github.com/liliyke/detection-tooling
+cd detection-tooling
+uv sync
+```
+
+If you're on plain pip:
+
+```bash
+python -m venv .venv && source .venv/bin/activate
+pip install -e ".[dev]"
+```
+
+## Usage pattern
+
+Each tool exposes a CLI named after the module:
+
+```bash
+uv run sigma-to-sentinel  --rule path/to/rule.yml  --out converted/
+uv run attack-coverage    --rules-dir ../detection-rules/rules  > navigator_layer.json
+uv run rule-validator     --rules-dir ../detection-rules/rules
+uv run telemetry-generator --template templates/lsass_access.yml --count 10 > events.jsonl
+uv run log-normalizer     --input /var/log/auth.log --source linux_auth > normalized.jsonl
+```
+
+`--help` on any one lists every flag.
+
+## Tests
+
+```bash
+uv run pytest -q
+```
+
+CI runs the same. Tests cover happy-path conversion, schema validation, and one round-trip per backend.
+
+## License
+
+MIT.

pyproject.toml

@@ -0,0 +1,37 @@
+[project]
+name = "detection-tooling"
+version = "0.1.0"
+description = "Small CLI utilities for operating Sigma-based detection content"
+requires-python = ">=3.10"
+dependencies = [
+    "pysigma>=0.11.0",
+    "pysigma-backend-kusto>=0.4.0",
+    "pysigma-backend-splunk>=1.1.0",
+    "pysigma-backend-elasticsearch>=1.1.0",
+    "pyyaml>=6.0",
+    "click>=8.1",
+    "rich>=13.7",
+]
+
+[project.optional-dependencies]
+dev = [
+    "pytest>=8.0",
+    "ruff>=0.5.0",
+]
+
+[project.scripts]
+sigma-to-sentinel    = "detection_tooling.sigma_to_sentinel:cli"
+attack-coverage      = "detection_tooling.attack_coverage:cli"
+rule-validator       = "detection_tooling.rule_validator:cli"
+telemetry-generator  = "detection_tooling.telemetry_generator:cli"
+log-normalizer       = "detection_tooling.log_normalizer:cli"
+
+[build-system]
+requires = ["setuptools>=68"]
+build-backend = "setuptools.build_meta"
+
+[tool.setuptools.packages.find]
+where = ["src"]
+
+[tool.pytest.ini_options]
+testpaths = ["tests"]

src/detection_tooling/__init__.py

@@ -0,0 +1,3 @@
+"""Detection engineering CLI tools."""
+
+__version__ = "0.1.0"

src/detection_tooling/attack_coverage.md

@@ -0,0 +1,75 @@
+# attack-coverage
+
+Build an [ATT&CK Navigator](https://mitre-attack.github.io/attack-navigator/) layer from a folder of Sigma rules. Useful for showing "here's what my detection content covers" in a single picture, and for spotting tactics where you're light.
+
+## What it does
+
+1. Walks every `.yml` under `--rules-dir`.
+2. Reads each rule's `tags:` list and pulls out `attack.tXXXX[.YYY]` entries.
+3. Tallies how many rules touch each technique.
+4. Writes a Navigator layer JSON. Score = rule count. Heatmap colors run from light blue (one rule) to dark blue (many).
+
+## Usage
+
+```bash
+# Print layer JSON to stdout
+uv run attack-coverage --rules-dir ../detection-rules/rules
+
+# Write to a file + print a table to stderr for quick inspection
+uv run attack-coverage --rules-dir ../detection-rules/rules \
+                      --out coverage.json \
+                      --table
+```
+
+Then drop `coverage.json` into the Navigator: open https://mitre-attack.github.io/attack-navigator/, click "Open Existing Layer → Upload from local", select the JSON.
+
+## Sample table output
+
+```
+                Technique coverage
+┏━━━━━━━━━━━┳━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┓
+┃ Technique ┃ Rules ┃ Files                          ┃
+┡━━━━━━━━━━━╇━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┩
+│ T1003     │ 2     │ lsass_memory_access_un...     │
+│ T1003.001 │ 2     │ lsass_memory_access_un...     │
+│ T1547.001 │ 1     │ registry_run_key_modif...     │
+│ T1218.011 │ 1     │ rundll32_unusual_parent.yml   │
+│ T1547     │ 0     │ —                             │
+└───────────┴───────┴───────────────────────────────┘
+```
+
+## How to read the Navigator output
+
+- **Solid colored cells** — techniques with at least one rule. Darker = more rules.
+- **Empty cells in a tactic column** — gaps. Worth a hunt design exercise to figure out whether you can cover them.
+- **Hover text** — shows the source rule filenames, so you can pivot back to the YAML.
+
+## Hooking into CI
+
+Commit `coverage.json` to your repo on every push so the file is always current:
+
+```yaml
+- name: Generate coverage layer
+  run: uv run attack-coverage --rules-dir rules --out coverage.json
+
+- name: Commit if changed
+  uses: stefanzweifel/git-auto-commit-action@v5
+  with:
+    commit_message: "Update ATT&CK coverage layer"
+    file_pattern: coverage.json
+```
+
+## Tuning ideas
+
+- **Two-color heatmap.** Add `--threshold N` so techniques with ≥N rules render in one color and others in another. Easier-to-read at a glance.
+- **Sub-technique aggregation.** Currently each subtechnique is its own row. Add a `--rollup` mode that aggregates a subtechnique's count into its parent.
+- **Per-platform layers.** Filter by `logsource.product` and emit one layer per platform — "Windows coverage", "Linux coverage", "cloud coverage".
+- **Diff mode.** Two `--rules-dir` flags. Output highlights techniques *gained* and *lost* between two repo states.
+- **Coverage-quality scoring.** Currently coverage is binary (any rule fires count). Add a `quality:` field to your Sigma rules (`level` could substitute) and weight the score — `critical=4, high=3, medium=2, low=1`.
+- **Combine with adversary-emulation results.** If you have a YAML mapping of "atomic tests we run", layer that too — green where you have *and* test, yellow where you have but don't test, red where you have neither.
+
+## References
+
+- [ATT&CK Navigator GitHub](https://github.com/mitre-attack/attack-navigator)
+- [Layer file format spec](https://github.com/mitre-attack/attack-navigator/blob/master/layers/LAYERFORMATv4_5.md)
+- [Detection-as-code: ATT&CK coverage tracking — SpecterOps](https://posts.specterops.io/)

src/detection_tooling/attack_coverage.py

@@ -0,0 +1,100 @@
+"""Build an ATT&CK Navigator layer JSON from a folder of Sigma rules.
+
+Walks every .yml under --rules-dir, harvests `attack.tXXXX[.YYY]` tags, then
+emits a Navigator layer where each technique's score is the count of rules
+covering it. Higher score = more redundancy / depth.
+"""
+
+from __future__ import annotations
+
+import json
+import re
+import sys
+from collections import Counter
+from pathlib import Path
+
+import click
+import yaml
+from rich.console import Console
+from rich.table import Table
+
+console = Console()
+
+TECH_RE = re.compile(r"^attack\.(t\d{4}(?:\.\d{3})?)$")
+
+
+def _iter_techniques(rules_dir: Path):
+    for p in rules_dir.rglob("*.yml"):
+        with p.open(encoding="utf-8") as f:
+            rule = yaml.safe_load(f) or {}
+        for tag in rule.get("tags", []) or []:
+            m = TECH_RE.match(tag)
+            if m:
+                yield m.group(1).upper(), p
+
+
+@click.command()
+@click.option("--rules-dir", type=click.Path(exists=True, path_type=Path), required=True,
+              help="Root directory containing Sigma .yml files (recurses).")
+@click.option("--name", default="Sigma Coverage",
+              help="Layer name shown in Navigator.")
+@click.option("--description", default="Generated from local Sigma rules",
+              help="Layer description.")
+@click.option("--out", "out_path", type=click.Path(path_type=Path), default=None,
+              help="Write JSON to this file. If omitted, prints to stdout.")
+@click.option("--table", is_flag=True,
+              help="Also print a coverage table to stderr.")
+def cli(rules_dir: Path, name: str, description: str, out_path: Path | None, table: bool) -> None:
+    """Output an ATT&CK Navigator layer JSON summarizing rule coverage."""
+    counts: Counter[str] = Counter()
+    by_tech: dict[str, list[str]] = {}
+
+    for tech, path in _iter_techniques(rules_dir):
+        counts[tech] += 1
+        by_tech.setdefault(tech, []).append(path.name)
+
+    layer = {
+        "name": name,
+        "versions": {"attack": "14", "navigator": "5.0.0", "layer": "4.5"},
+        "domain": "enterprise-attack",
+        "description": description,
+        "techniques": [
+            {
+                "techniqueID": tech,
+                "score": cnt,
+                "color": "",
+                "comment": "; ".join(by_tech[tech]),
+                "enabled": True,
+            }
+            for tech, cnt in counts.most_common()
+        ],
+        "gradient": {
+            "colors": ["#a4d3f5", "#1e6091"],
+            "minValue": 1,
+            "maxValue": max(counts.values()) if counts else 1,
+        },
+        "legendItems": [],
+        "showTacticRowBackground": False,
+        "tacticRowBackground": "#dddddd",
+        "selectTechniquesAcrossTactics": True,
+    }
+
+    output = json.dumps(layer, indent=2)
+    if out_path:
+        out_path.write_text(output, encoding="utf-8")
+        console.print(f"[green]wrote[/green] {out_path}", file=sys.stderr)
+    else:
+        sys.stdout.write(output + "\n")
+
+    if table:
+        t = Table(title="Technique coverage")
+        t.add_column("Technique")
+        t.add_column("Rules", justify="right")
+        t.add_column("Files")
+        for tech, cnt in counts.most_common():
+            t.add_row(tech, str(cnt), ", ".join(by_tech[tech]))
+        console.print(t, file=sys.stderr)
+
+
+if __name__ == "__main__":
+    cli()