diff --git a/README.md b/README.md index 355c717..cbe7e7b 100644 --- a/README.md +++ b/README.md @@ -21,6 +21,21 @@ it was tested, and which beacons didn't answer. Use it as a web app, a CLI, a batch log scanner, a headless log watcher or an MCP server for AI assistants. Where it's headed: [ROADMAP.md](ROADMAP.md), on unmasking borrowed legitimacy. +## Status and known limits + +This is a working beta: a triage aid for an analyst to check, not something to act on without a person. Today: + +- **Accuracy rests on a small set.** Calibration uses 55 labeled IPs. Most paradoxes on known-malicious IPs fall away + once VirusTotal is on, as [the note on the results](#calibration) explains. +- **Evidence from OTX alone can be gamed.** In synthetic tests, 5 new OTX accounts take a clean IP to Likely malicious, + and OTX's own whitelist or padded pulses can hide real OTX evidence. Independent beacons hold up against the same + attacks. See [the adversarial results](ROADMAP.md#35-adversarial-test-suite--m). +- **Not yet tested:** use by real analysts, prompt injection against the summary, and attackers hiding on CDNs, cloud + hosts or Tor. +- **VirusTotal's free tier** rules out commercial use, and its limits make a 25-IP scan take at least 6 minutes. + +The threat model and its open gaps are in [SECURITY.md](SECURITY.md). + ## Install ```bash @@ -469,6 +484,7 @@ paradox/cli.py the `paradox` command: app + headless subcommands paradox/sources.py beacon adapters -> Signal paradox/scoring.py tier scores, paradox, threat paradox/engine.py concurrent scan + caching +paradox/quota.py keeps VirusTotal inside its free limits paradox/history.py SQLite scan history + lag measurement paradox/logscan.py batch mode: extract IPs from logs, scan, rank paradox/watch.py headless sensor: follow logs, alert, periodic rescans diff --git a/ROADMAP.md b/ROADMAP.md index 50e1057..a699881 100644 --- a/ROADMAP.md +++ b/ROADMAP.md @@ -24,13 +24,14 @@ Effort: **S** is about a day, **M** a few days, **L** a week or more. Each item has a permanent story ID (`PDX-n`) and a "done when" line for the [Muster board](https://github.com/Castor-o7/muster/blob/main/docs/board.md). A checked box means shipped, not verified. -The watchman keeps each story's state. +The watchman keeps each story's state. **Progress: 21 of 40 stories shipped (52%),** counting each story the same +whatever its size, and the total grows as work finds new stories. --- ## Where we stand -What the 2026-10-29 calibration (run 5) and the live tests showed, which sets the priorities below. A ✅ marks a shipped fix; the rescore after 1.1, 1.2 and 3.2 is in the README: +What the 2026-10-29 calibration (run 5), the live tests and later measurements showed, which sets the priorities below. A ✅ marks a shipped fix; the rescore after 1.1, 1.2 and 3.2 is in the README: | Finding | Evidence | Addressed by | |---|---|---| @@ -40,6 +41,8 @@ What the 2026-10-29 calibration (run 5) and the live tests showed, which sets th | Blocklists used as labels include research scanners | Shodan and Palo Alto Xpanse IPs were in the "malicious" set | 3.2 ✅ | | We assume which beacons are fast and slow, and never measured it | Tiers are hard-coded in `BEACONS` | 3.4 | | Veneer has no ground truth, so we can't state its precision | No labeled veneer cases exist | 3.3 | +| Evidence from OTX alone can be gamed | Synthetic attacks: 5 new accounts take a clean IP to Likely malicious; OTX's whitelist or padding can hide real evidence | 3.5 ✅ measured; 3.8, 3.9, 3.10 to fix | +| Most paradoxes on malicious IPs reflected the missing VirusTotal key | With it on, malicious at paradox 35+ fell from 28 to 4 of 30 | Open: how to frame the score | ---