Fix WiFi frame subtype bitmask bug + add missing OUI prefixes#13
Fix WiFi frame subtype bitmask bug + add missing OUI prefixes#13dougborg wants to merge 23 commits into
Conversation
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Provides per-variant build/upload/flash/monitor targets for all 6 hardware variants, plus LittleFS data upload for variants with audio assets. Uses GNU Make define/eval/call to generate targets from a single template. Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Tests compile natively with clang++/g++ — no ESP32 hardware needed. Covers detectors, device tracker state machine, and threat analyzer scoring pipeline (37 cases, 126 assertions) targeting the M5Stick variant's pure-logic headers. Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Move DetectorTypes.h, Detectors.h, DeviceSignatures.h, EventBus.h, ThreatAnalyzer.h, and TelemetryReporter.h from the M5Stick variant's src/ into a new top-level common/ directory. Update Makefile to pass -I common via build.extra_flags for all variants, update test includes, and fix the M5Stick FQBN (m5stick_c_plus2 -> m5stack_stickc_plus2). Merge AudioEvent (from M5Fire's EventBus.h) into the shared header so other variants can adopt it when migrated. Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Delete local copies of EventBus.h, DeviceSignatures.h, ThreatAnalyzer.h, and TelemetryReporter.h from m5fire/src/. Replace legacy ThreatAnalyzer (simple boolean matching) and TelemetryReporter (DynamicJsonDocument with nested objects) implementations with the shared detector-based system. Add ISR-safe deferred event processing with portMUX spinlocks for WiFi, BLE, and threat events. Add ThreatAnalyzer::tick() heartbeat in loop() and shouldAlert gate on triggerAlert(). Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Delete local copies of EventBus.h, DeviceSignatures.h, ThreatAnalyzer.h, and TelemetryReporter.h from mini12864/src/. Replace legacy ThreatAnalyzer and TelemetryReporter implementations with shared detector-based system. Add ISR-safe deferred event processing with portMUX spinlocks. Move display notifications (Mini12864DisplayNotifyWifiFrame, ShowAlert) and audio playback to the main loop's deferred handlers. Add tick() heartbeat and shouldAlert gate. Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Also fix Makefile build.extra_flags override that was clobbering ESP32 core defines (-DESP32=ESP32 etc). Use build.defines instead, which is included within build.extra_flags and starts empty. Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Move variant-specific src/ files into sketch directory to follow Arduino convention. Keep Flipper's own TelemetryReporter (line-based protocol for Flipper app). Fix radioType null-check for char array. Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Dockerfile builds a debian:bookworm-slim image containing arduino-cli, ESP32 core v3.0.7, all Arduino libraries (version-pinned), doctest.h, and pre-warmed core caches for all 4 FQBNs. Source is bind-mounted at runtime so the image is reusable across branches. Also adds docker-compose.yml (build-all, test, shell, build-variant services), entrypoint.sh (seeds doctest.h into bind-mount), .dockerignore, and Makefile docker-* targets. Fixes a portability bug in test/mocks/Arduino.h: adds <cstdio> for snprintf, which macOS clang resolves transitively but Debian clang-14 does not. Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Audit all dependency versions against latest available in the Arduino library index, update where possible, and centralize pins in a single versions.env file consumed by both Makefile (include) and Dockerfile (--build-arg). Version changes: - Base image: debian:bookworm-slim → debian:trixie-slim (Debian 13) - arduino-cli: unpinned → 1.4.1 - ArduinoJson: 7.3.0 → 7.4.2 - NimBLE-Arduino: 2.2.1 → 2.3.7 - M5Unified: 0.2.2 → 0.2.11 - Adafruit SSD1306: 2.5.13 → 2.5.16 - doctest: 2.4.11 → 2.4.12 - ESP32 core: 3.0.7 (unchanged — newer causes IRAM overflow) - U8g2: 2.35.30 (unchanged — already latest in Arduino index) - Adafruit GFX: 1.12.4 (unchanged — already latest) Makefile install-deps now pins library versions from versions.env, matching what the Dockerfile installs. Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Move duplicated content (setup, architecture, telemetry, configuration, troubleshooting, extending) from 6 variant READMEs into 8 shared docs/ files. Add new docs for build system and testing (previously undocumented). Restructure CLAUDE.md as a scannable agent gateway with dispatch table. Fix incorrect .ino filename reference in portable variant README. Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Make RadioScanner timing values dynamic across all variants by replacing static const members with static members initialized to battery-mode defaults. Add setPerformanceMode() to switch between battery and external-power scanning parameters. M5Stick and M5Fire main loops now detect external power via M5.Power.isCharging() and automatically switch to aggressive scanning (200ms dwell, 3s BLE scan, 4s interval) and force the display awake. Non-M5 variants get the API but remain at battery-mode defaults. Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
NimBLE GATT server streams newline-delimited JSON telemetry to the DeFlock app over BLE, enabling iOS support and wireless operation on Android. When a BLE client connects, scan duty is reduced to share radio time; when it disconnects (e.g. USB takeover), full scan duty resumes. - Add common/BleTransport.h: GATT server with notify characteristic, MTU negotiation up to 512, chunked notifications, client state callback - Modify TelemetryReporter.h: dual output to Serial + BLE, with buffer overflow protection and truncation detection - Modify RadioScanner.h: adaptive scan duty cycle based on BLE client state, deferred to main loop for thread safety - Wire BleTransport into M5StickC setup, guard NimBLE double-init Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Battery readings from M5.Power.getBatteryLevel() jitter at charge boundaries (e.g. 79%<->80%). Replace raw reads with a rolling median of the last 8 samples (taken every 3s) to stabilize the display. The filter is extracted to common/BatterySmoothing.h so the median logic is host-testable with doctest. Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
The mask 0x0F selects bits 0-3 (protocol version + frame type), which is always 0 for management frames, silently breaking all WiFi subtype detection. The correct mask 0x00F0 selects bits 4-7 (subtype field). All other 5 variants already used the correct mask. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Add 7 Flock WiFi OUI prefixes (Liteon/USI contract manufacturers) field-confirmed and IEEE cross-referenced from upstream reports: f4:6a:dd, f8:a2:d6, e0:0a:f6, 00:f4:8d, d0:39:57, e8:d0:fc, 24:b2:b9 Add SoundThinking (ShotSpotter) BLE OUI d4:11:d6 to surveillance prefixes for acoustic gunshot detection sensor awareness. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
There was a problem hiding this comment.
Pull request overview
This PR fixes WiFi 802.11 management frame subtype parsing (restoring probe/beacon detection), expands the device signature database with additional OUIs, and introduces a shared common/ core plus a reproducible build/test toolchain (Makefile + Docker + host-side unit tests).
Changes:
- Correct WiFi frame subtype bitmask usage and add probe response handling in WiFi sniffers.
- Add new MAC OUI prefixes (Flock-related + surveillance vendors) and new detector/alert-tier plumbing.
- Add build/test infrastructure (
versions.env, Makefile, Docker) and host-side doctest unit tests; migrate shared logic intocommon/.
Reviewed changes
Copilot reviewed 73 out of 76 changed files in this pull request and generated 6 comments.
Show a summary per file
| File | Description |
|---|---|
| versions.env | Centralizes pinned dependency versions for Makefile/Docker. |
| entrypoint.sh | Docker entrypoint to seed doctest.h into bind-mounted workspace. |
| docker-compose.yml | Adds compose services for build/test/shell. |
| Dockerfile | Provides pre-baked toolchain + libraries for reproducible builds. |
| Makefile | Adds arduino-cli build/upload + host-side doctest test targets. |
| .gitignore | Ignores build output and fetched doctest.h. |
| .dockerignore | Reduces Docker build context size. |
| CLAUDE.md | Project conventions/architecture notes (developer-facing). |
| README.md | Updates top-level docs to reflect variants + new documentation structure. |
| docs/getting-started.md | New shared setup guide for variants. |
| docs/architecture.md | New architecture overview (pipeline, detectors, thread safety). |
| docs/build-system.md | New Makefile/Docker build documentation. |
| docs/configuration.md | New configuration guide for scan/detector tuning. |
| docs/telemetry-format.md | New JSON + UART telemetry reference. |
| docs/testing.md | New host-side testing documentation. |
| docs/extending.md | New guide for adding detectors/patterns/variants. |
| docs/troubleshooting.md | New common troubleshooting guide. |
| common/DetectorTypes.h | Introduces detector flags, alert levels, tracking constants/types. |
| common/EventBus.h | Extends ThreatEvent fields; defines shared event structures/APIs. |
| common/DeviceSignatures.h | Adds OUI tables (Flock + surveillance) and computed counts. |
| common/Detectors.h | Implements detector functions (SSID/OUI/BLE UUID/name + RSSI modifier). |
| common/ThreatAnalyzer.h | Adds detector registry, alert-tier logic, device tracking, heartbeat. |
| common/TelemetryReporter.h | Implements JSON telemetry (and optional BLE transport hook). |
| common/BleTransport.h | Adds NimBLE GATT server for streaming JSON telemetry to a client. |
| common/BatterySmoothing.h | Adds rolling-median battery percentage smoothing filter. |
| common/ConnectionStatus.h | Adds pure helpers for serial/battery trend state. |
| test/test_main.cpp | Adds doctest main entry point. |
| test/mocks/Arduino.h | Adds minimal Arduino API mock for host builds. |
| test/eventbus_impl.cpp | Provides EventBus storage + mock millis() for tests. |
| test/test_detectors.cpp | Unit tests for detector helpers and detectors. |
| test/test_device_tracker.cpp | Unit tests for DeviceTracker state machine + eviction. |
| test/test_connection_status.cpp | Unit tests for connection/battery helpers. |
| test/test_battery_smoothing.cpp | Unit tests for rolling median battery smoothing. |
| m5stack/flocksquawk_m5stick/src/RadioScanner.h | Adds performance/duty-cycle controls and runtime scan parameter tuning. |
| m5stack/flocksquawk_m5stick/src/EventBus.h | Removed (variant now uses shared headers). |
| m5stack/flocksquawk_m5stick/src/DeviceSignatures.h | Removed (variant now uses shared signatures). |
| m5stack/flocksquawk_m5stick/src/ThreatAnalyzer.h | Removed (variant now uses shared analyzer). |
| m5stack/flocksquawk_m5stick/src/TelemetryReporter.h | Removed (variant now uses shared reporter). |
| m5stack/flocksquawk_m5stick/flocksquawk_m5stick.ino | Updates includes + integrates BLE transport + deferred event processing + subtype mask fix. |
| m5stack/flocksquawk_m5stick/README.md | Refactors/links docs to shared documentation and common headers. |
| m5stack/flocksquawk_m5fire/src/RadioScanner.h | Adds runtime scan parameter tuning and performance mode. |
| m5stack/flocksquawk_m5fire/src/EventBus.h | Removed (variant now uses shared headers). |
| m5stack/flocksquawk_m5fire/src/DeviceSignatures.h | Removed (variant now uses shared signatures). |
| m5stack/flocksquawk_m5fire/src/ThreatAnalyzer.h | Removed (variant now uses shared analyzer). |
| m5stack/flocksquawk_m5fire/src/TelemetryReporter.h | Removed (variant now uses shared reporter). |
| m5stack/flocksquawk_m5fire/flocksquawk_m5fire.ino | Fixes subtype mask; adds deferred event processing + power-based performance mode. |
| m5stack/flocksquawk_m5fire/README.md | Refactors/links docs to shared documentation and common headers. |
| Mini12864/flocksquawk_mini12864/src/RadioScanner.h | Adds runtime scan parameter tuning and performance mode. |
| Mini12864/flocksquawk_mini12864/src/DeviceSignatures.h | Removed (variant now uses shared signatures). |
| Mini12864/flocksquawk_mini12864/src/ThreatAnalyzer.h | Removed (variant now uses shared analyzer). |
| Mini12864/flocksquawk_mini12864/src/TelemetryReporter.h | Removed (variant now uses shared reporter). |
| Mini12864/flocksquawk_mini12864/flocksquawk_mini12864.ino | Fixes subtype mask; adds deferred event processing + heartbeat + alert gating. |
| 128x32_OLED/flocksquawk_128x32/src/RadioScanner.h | Adds runtime scan parameter tuning and performance mode. |
| 128x32_OLED/flocksquawk_128x32/src/EventBus.h | Removed (variant now uses shared headers). |
| 128x32_OLED/flocksquawk_128x32/src/DeviceSignatures.h | Removed (variant now uses shared signatures). |
| 128x32_OLED/flocksquawk_128x32/src/ThreatAnalyzer.h | Removed (variant now uses shared analyzer). |
| 128x32_OLED/flocksquawk_128x32/src/TelemetryReporter.h | Removed (variant now uses shared reporter). |
| 128x32_OLED/flocksquawk_128x32/flocksquawk_128x32.ino | Fixes subtype bitmask; adds probe response parsing and deferred event processing. |
| 128x32_OLED/flocksquawk_128x32_portable/src/RadioScanner.h | Adds runtime scan parameter tuning and performance mode. |
| 128x32_OLED/flocksquawk_128x32_portable/src/EventBus.h | Removed (variant now uses shared headers). |
| 128x32_OLED/flocksquawk_128x32_portable/src/DeviceSignatures.h | Removed (variant now uses shared signatures). |
| 128x32_OLED/flocksquawk_128x32_portable/src/ThreatAnalyzer.h | Removed (variant now uses shared analyzer). |
| 128x32_OLED/flocksquawk_128x32_portable/src/TelemetryReporter.h | Removed (variant now uses shared reporter). |
| 128x32_OLED/flocksquawk_128x32_portable/flocksquawk_128x32_portable.ino | Fixes subtype mask; adds deferred event processing + heartbeat + alert gating. |
| 128x32_OLED/flocksquawk_128x32_portable/README.md | Refactors/links docs to shared documentation and common headers. |
| flipper-zero/dev-board-firmware/src/EventBus.h | Removed (variant now uses shared headers). |
| flipper-zero/dev-board-firmware/src/DeviceSignatures.h | Removed (variant now uses shared signatures). |
| flipper-zero/dev-board-firmware/src/ThreatAnalyzer.h | Removed (variant now uses shared analyzer). |
| flipper-zero/dev-board-firmware/flocksquawk-flipper/src/RadioScanner.h | Adds runtime scan parameter tuning and performance mode. |
| flipper-zero/dev-board-firmware/flocksquawk-flipper/src/TelemetryReporter.h | Adds Flipper-specific UART reporter header. |
| flipper-zero/dev-board-firmware/flocksquawk-flipper/src/SoundEngine.h | Adds Flipper “no audio” stub header. |
| flipper-zero/dev-board-firmware/flocksquawk-flipper/flocksquawk-flipper.ino | Fixes subtype mask; adds deferred event processing and safer telemetry formatting. |
| flipper-zero/README.md | Refactors/links docs to shared documentation and common headers. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| | `target.category` | string | `"surveillance_device"` or `"acoustic_detector"` | | ||
| | `target.should_alert` | boolean | `true` on first detection above threshold | | ||
| | `target.detectors` | object | Which detectors fired and their weights | | ||
|
|
||
| ### Detector Keys | ||
|
|
||
| The `detectors` object only includes detectors that matched. Possible keys: | ||
|
|
||
| | Key | Meaning | Typical weight | | ||
| |-----|---------|---------------| | ||
| | `ssid_format` | Exact SSID pattern match | 75 | | ||
| | `ssid_keyword` | SSID keyword substring match | 45 | | ||
| | `mac_oui` | MAC OUI prefix match | 20 | | ||
| | `ble_name` | BLE device name match | 55 | | ||
| | `raven_custom_uuid` | Raven custom UUID match | 80 | | ||
| | `raven_std_uuid` | Raven standard UUID match | 10 | | ||
| | `rssi_modifier` | RSSI proximity adjustment | -10 to +10 | | ||
|
|
There was a problem hiding this comment.
Telemetry docs don’t match the current implementation: ThreatAnalyzer can emit category "surveillance_camera" (not listed here), and TelemetryReporter currently always includes rssi_modifier in matchFlags even when it’s 0 (so the statement that the detectors object “only includes detectors that matched” is inaccurate). Please update this section to reflect the actual category values and when rssi_modifier is present.
| # FlockSquawk — arduino-cli build system | ||
| # Usage: make help | ||
|
|
||
| # ────────────────────────────────────────────── | ||
| # Dependency versions (shared with Dockerfile) | ||
| # ────────────────────────────────────────────── | ||
| include versions.env | ||
|
|
||
| # ────────────────────────────────────────────── | ||
| # User-configurable variables | ||
| # ────────────────────────────────────────────── | ||
| PORT ?= |
There was a problem hiding this comment.
The PR title/description focuses on a WiFi subtype bitmask fix + OUI additions, but this change set also introduces a new Makefile/Docker build system, a common/ shared-header architecture, and substantial refactors across multiple variants and docs. Please either update the PR title/description to reflect the full scope or split the build-system/architecture refactor into a separate PR to keep review/rollback risk manageable.
| private: | ||
| unsigned long bootTime; | ||
| BleTransport* _bleTransport = nullptr; | ||
|
|
||
| // Defined in .ino after BleTransport.h is included | ||
| inline void _sendViaBle(char* buf, size_t len); | ||
| }; |
There was a problem hiding this comment.
TelemetryReporter declares _sendViaBle() but doesn’t provide a default implementation. Only m5stick/flocksquawk_m5stick.ino defines it; other variants that include TelemetryReporter.h will fail to link because handleThreatDetection() calls _sendViaBle() when _bleTransport is set. Provide a no-op default implementation in TelemetryReporter (e.g., guarded by #if when BLE is unavailable) or add the same definition to every variant that includes this header.
| Each variant is a self-contained Arduino sketch. Open the `.ino` file for your variant directly in the Arduino IDE: | ||
|
|
There was a problem hiding this comment.
This guide states each variant is a “self-contained Arduino sketch”, but the sketches now include shared headers (e.g., EventBus.h, ThreatAnalyzer.h) from common/ via the Makefile’s -I common. Open-in-Arduino-IDE builds will fail unless users copy/symlink the shared headers into the sketch (or the includes are changed to relative paths). Please update this section to reflect the required workflow.
| static const char* const keywords[] = { | ||
| "flock", "penguin", "pigvision", "test_flck" | ||
| }; | ||
| static const uint8_t count = sizeof(keywords) / sizeof(keywords[0]); |
There was a problem hiding this comment.
detectSsidKeyword() includes a special-case keyword ("test_flck") that appears to exist only to satisfy unit tests. Shipping test-only keywords in production detection logic can cause unintended false positives. Prefer updating the tests to use a realistic keyword-only SSID (e.g., one that matches "flock" but not the exact format detector) and remove "test_flck" from the production keyword list.
| Edit `src/RadioScanner.h` in your variant: | ||
|
|
||
| ```cpp | ||
| static const uint8_t MAX_WIFI_CHANNEL = 13; | ||
| static const uint16_t CHANNEL_SWITCH_MS = 500; | ||
| ``` | ||
|
|
||
| - `MAX_WIFI_CHANNEL` -- highest channel to scan (1-13, or 1-11 for US-only) | ||
| - `CHANNEL_SWITCH_MS` -- dwell time per channel in milliseconds (500 in most variants, 1000 in M5Stick) | ||
|
|
||
| ## BLE Scan Interval | ||
|
|
||
| Edit `src/RadioScanner.h` in your variant: | ||
|
|
||
| ```cpp | ||
| static const uint8_t BLE_SCAN_SECONDS = 1; | ||
| static const uint32_t BLE_SCAN_INTERVAL_MS = 5000; | ||
| ``` |
There was a problem hiding this comment.
This section/documentation shows CHANNEL_SWITCH_MS, BLE_SCAN_SECONDS, and BLE_SCAN_INTERVAL_MS as static const compile-time constants in src/RadioScanner.h, but the code now defines them as mutable static variables (and in several variants they’re initialized/changed at runtime via setPerformanceMode() / duty-cycle logic). Please update the snippets and explanation so they match how the parameters are actually configured now.
Summary
0x0F→0x00F0) that silently broke all WiFi subtype detection — probe requests, probe responses, and beacons were never matched. All 6 variants now use the correct mask.f4:6a:dd,f8:a2:d6,e0:0a:f6,00:f4:8d,d0:39:57,e8:d0:fc,24:b2:b9) from Liteon/USI contract manufacturers, field-confirmed and IEEE cross-referenced via flock-you PR #29 and Issue #28.d4:11:d6) to surveillance prefixes for acoustic gunshot detection sensor awareness.Test plan
make test— all 75 host-side tests pass.inofiles use0x00F0maskMACPrefixCount/SurveillancePrefixCountauto-compute viasizeof(no hardcoded counts)🤖 Generated with Claude Code