Hardware-in-the-loop regression testing for Flipper Zero firmware. Two Flippers, one air gap, a pass/fail result.
Both directions: every case runs with the candidate firmware receiving (known-good emitter → DUT) and transmitting (DUT → known-good emitter), so a regression in either the decoder or the encoder/emulator is caught.
Flipper's integration test cases are run by hand today. You can't mock radio in software, so this project uses a second physical device instead. One Flipper (the emitter) sends a known signal. The other (the DUT, running the candidate firmware unmodified) receives and decodes it. A Python/pytest runner checks the decode against a versioned fixture and writes JUnit XML that any CI system can read.
┌────────────────┐ USB serial CLI ┌───────────────────────┐
│ │────────────────────▶│ EMITTER (Flipper #2) │
│ HOST RUNNER │ └──────────┬────────────┘
│ Python+pytest │ │ Sub-GHz / IR / RFID / NFC
│ │ air gap / wire │ (spacing fixed by a jig)
│ │ USB serial CLI ▼
│ │────────────────────▶┌───────────────────────┐
│ │◀──── decoded ───────│ DUT (candidate fw) │
└───────┬────────┘ │ stock CLI, no test fw │
│ JUnit XML └───────────────────────┘
▼
PR status check (self-hosted bench runner)
| Subsystem | Counterpart | Asserts | Fit | Fixtures |
|---|---|---|---|---|
| Sub-GHz | 2nd Flipper subghz tx / tx_from_file |
protocol, bits, key, frequency | strong | 6 |
| Infrared | 2nd Flipper ir tx |
protocol, address, command | strong | 8 |
| iButton | 2nd Flipper ikey emulate, wired 1-Wire |
key type, ID | strong | 2 |
| RFID 125 kHz | 2nd Flipper rfid emulate (ASK, FSK, PSK) |
protocol, data | strong | 4 |
| GPIO | 2nd Flipper gpio set, wired |
pin level | strong | 2 |
| NFC 13.56 MHz | 2nd Flipper nfc emulate (.nfc upload) |
tag type, UID | partial | 1 |
| BadUSB | host captures HID via evdev | typed text | partial, operator-assisted | 1 |
| Screen / UI | none | none | manual, out of scope | none |
Every serial command is checked against the firmware source (file references are in src/flci/cli.py). Nothing has been run on a physical bench yet. Bench reports are the most useful contribution right now (see Contributing).
- Stock-firmware DUT: driven only over the USB CLI; no test build, no SWD.
- 29 fixtures across 7 subsystems, all YAML. Adding a case needs no code.
- Receive and transmit paths: each fixture runs as
@dut_rxand@dut_tx(47 hardware cases in total). - Negative fixtures:
silencecases listen with nothing sent and must decode nothing, which catches ghost decodes and stuck decoders. - Timing budgets: a fixture can set
max_duration_s; a correct but slow decode fails.flci soaksuggests budgets from measured p95. flci doctor: checks ports, permissions, qFlipper, roles, firmware CLI commands, SD card, battery and (with--probe) wiring, and says exactly what to fix.- HTML reports:
flci report ... --html report.htmlwrites one self-contained, shareable page (light/dark). - Preflight: if the DUT firmware lacks a needed CLI command, or its region blocks a frequency, the test is skipped with the reason instead of reported as a decode failure (
FLCI_STRICT=1turns those into failures). - Flakiness measurement:
flci soak --runs 20gives per-fixture pass rates, p50/p95 timings, every distinct decode, and the RFC evidence table. - Regression gate:
flci compare base.xml candidate.xmllists what broke, what got fixed and what lost coverage. CI compares every run against the last greenmain. - Battery logging: each result records both devices' charge level, since low charge shortens RF/RFID range.
- CI: self-hosted bench workflow, DUT flashing via
update install, JUnit check run plus a Markdown job summary. - Hardware: parametric OpenSCAD jig (STL rendered in CI) and a wiring guide.
git clone https://github.com/Krishita17/flipper-loopback
cd flipper-loopback
python3 -m venv .venv && source .venv/bin/activate
pip install -e '.[dev]' # add ,badusb on Linux for HID captureWith no hardware, the parser and orchestration tests run and the hardware tests are skipped with a reason. A skip is never reported as a pass:
pytestWith two Flippers on USB (close qFlipper first, since it holds the serial port):
flci doctor # is the bench ready? what to fix
export FLCI_DUT=<port or device name> # flashed with the build under test
export FLCI_EMITTER=<port or device name> # known-good counterpart
export FLCI_CAPABILITIES=subghz,infrared,rfid,nfc # what your bench is set up for
pytest -m hardware -v # JUnit XML -> reports/junit.xml| Variable | Meaning |
|---|---|
FLCI_DUT, FLCI_EMITTER |
Required. Port path or Flipper name. Roles are never guessed |
FLCI_CAPABILITIES |
Subsystems this bench supports. Default subghz,infrared,rfid,nfc; add ibutton, gpio once wired, and badusb for operator runs |
FLCI_DIRECTIONS |
dut_rx, dut_tx or both (default both) |
FLCI_TAGS |
Only run fixtures with these tags, e.g. smoke, negative |
FLCI_EXCLUDE_TAGS |
Leave out fixtures with these tags (default environment) |
FLCI_OPERATOR=1 |
Allow the operator-assisted BadUSB test (run with pytest -s) |
FLCI_STRICT=1 |
Treat missing CLI commands and region locks as failures instead of skips |
A failing round trip prints what was expected, every packet the DUT decoded, both firmware versions, and the tail of the DUT's raw output.
| Command | What it does |
|---|---|
flci doctor [--probe] |
Check the bench is ready; --probe runs one smoke case per subsystem |
flci detect |
List connected Flippers, firmware versions and assigned roles |
flci fixtures |
List and validate every fixture |
flci soak --runs 20 [--subsystem rfid] [--tag smoke] [--direction dut_tx] |
Repeat cases, write reports/soak.{json,md} with suggested timing budgets |
flci report reports/junit.xml [--html out.html] |
Markdown summary of a run (or --soak reports/soak.json), optional HTML page |
flci compare base.xml cand.xml |
Regressions between two runs; exit code 1 if any |
flci record from-sub/from-ir/live ... |
Create a fixture from a saved file or a live capture |
flci flash PACKAGE |
Flash the DUT from an update package and verify it |
Comparing two firmware builds on one bench:
flci flash old-update.tgz && pytest -m hardware --junitxml=reports/base.xml
flci flash new-update.tgz && pytest -m hardware --junitxml=reports/cand.xml
flci compare reports/base.xml reports/cand.xmlEach test case is a YAML file in fixtures/<subsystem>/, sometimes with a stimulus file (.sub, .nfc, BadUSB .txt) next to it. Adding a case doesn't need any code changes:
id: ir_nec_04_08
subsystem: infrared
stimulus: { kind: message, protocol: NEC, address: "0x04", command: "0x08", bursts: 3 }
expected: { subsystem: infrared, protocol: NEC, payload: "04-08" }
tags: [smoke]You can also record fixtures from real captures or saved Flipper files:
flci record from-sub my_remote.sub --id gate_433
flci record from-ir tv.ir --signal Power --id tv_power
flci record live rfid --port "$FLCI_DUT" --id my_fobNegative cases and timing budgets are fields on the same YAML:
id: rfid_silence
subsystem: rfid
stimulus: { kind: silence, read_timeout_s: 4 } # send nothing
expect_no_decode: true # any decode fails the case
max_duration_s: 8 # optional timing budgetA broken fixture (bad YAML, unknown subsystem, missing stimulus file, duplicate id, both or neither of expected/expect_no_decode) stops the run with the file name and the reason.
.github/workflows/hil.yml runs the suite on a self-hosted runner with both Flippers attached, optionally flashes the DUT first (flci flash, stock update install, no DFU), publishes JUnit as a HIL results check, writes a Markdown job summary and an HTML report artifact, and compares against the last green main run to flag regressions. It starts with flci doctor, so a misconfigured bench fails fast with a clear reason. A manual run can also soak every fixture N times. It runs one job at a time (there's only one bench) and never runs fork PRs. Setup is in docs/CI.md.
- hardware/jig/: parametric OpenSCAD jig (back-to-back for RF/RFID/NFC, head-to-head for IR). CI renders the STLs.
- hardware/WIRING.md: GPIO and iButton jumpers.
Architecture · Roadmap & nondeterminism log · CI setup · Upstream RFC
- Only use hardware and signals you own.
- Keep TX power low and use shielding or short distances so a bench rig doesn't transmit onto other people's devices or licensed bands. Follow your region's rules. The firmware enforces region locks and the harness treats a refused TX as an error. It never works around one.
- This project checks decode correctness of firmware. It is not a way to defeat any security system. Fixture keys and IDs are synthetic test values.
The most useful thing right now is a bench report: run flci soak --runs 20 and open an issue with the generated reports/soak.md, your firmware versions and jig spacing. New fixtures and CLI fixes are welcome too (see CONTRIBUTING.md).
MIT