Skip to content

Repository files navigation

flipper-loopback

Hardware-in-the-loop regression testing for Flipper Zero firmware. Two Flippers, one air gap, a pass/fail result.

CI python license bench

Both directions: every case runs with the candidate firmware receiving (known-good emitter → DUT) and transmitting (DUT → known-good emitter), so a regression in either the decoder or the encoder/emulator is caught.

Flipper's integration test cases are run by hand today. You can't mock radio in software, so this project uses a second physical device instead. One Flipper (the emitter) sends a known signal. The other (the DUT, running the candidate firmware unmodified) receives and decodes it. A Python/pytest runner checks the decode against a versioned fixture and writes JUnit XML that any CI system can read.

┌────────────────┐   USB serial CLI    ┌───────────────────────┐
│                │────────────────────▶│ EMITTER  (Flipper #2) │
│  HOST RUNNER   │                     └──────────┬────────────┘
│  Python+pytest │                                │ Sub-GHz / IR / RFID / NFC
│                │                  air gap / wire │ (spacing fixed by a jig)
│                │   USB serial CLI               ▼
│                │────────────────────▶┌───────────────────────┐
│                │◀──── decoded ───────│ DUT  (candidate fw)   │
└───────┬────────┘                     │ stock CLI, no test fw │
        │ JUnit XML                    └───────────────────────┘
        ▼
   PR status check (self-hosted bench runner)

Coverage

Subsystem Counterpart Asserts Fit Fixtures
Sub-GHz 2nd Flipper subghz tx / tx_from_file protocol, bits, key, frequency strong 6
Infrared 2nd Flipper ir tx protocol, address, command strong 8
iButton 2nd Flipper ikey emulate, wired 1-Wire key type, ID strong 2
RFID 125 kHz 2nd Flipper rfid emulate (ASK, FSK, PSK) protocol, data strong 4
GPIO 2nd Flipper gpio set, wired pin level strong 2
NFC 13.56 MHz 2nd Flipper nfc emulate (.nfc upload) tag type, UID partial 1
BadUSB host captures HID via evdev typed text partial, operator-assisted 1
Screen / UI none none manual, out of scope none

Every serial command is checked against the firmware source (file references are in src/flci/cli.py). Nothing has been run on a physical bench yet. Bench reports are the most useful contribution right now (see Contributing).

Features

  • Stock-firmware DUT: driven only over the USB CLI; no test build, no SWD.
  • 29 fixtures across 7 subsystems, all YAML. Adding a case needs no code.
  • Receive and transmit paths: each fixture runs as @dut_rx and @dut_tx (47 hardware cases in total).
  • Negative fixtures: silence cases listen with nothing sent and must decode nothing, which catches ghost decodes and stuck decoders.
  • Timing budgets: a fixture can set max_duration_s; a correct but slow decode fails. flci soak suggests budgets from measured p95.
  • flci doctor: checks ports, permissions, qFlipper, roles, firmware CLI commands, SD card, battery and (with --probe) wiring, and says exactly what to fix.
  • HTML reports: flci report ... --html report.html writes one self-contained, shareable page (light/dark).
  • Preflight: if the DUT firmware lacks a needed CLI command, or its region blocks a frequency, the test is skipped with the reason instead of reported as a decode failure (FLCI_STRICT=1 turns those into failures).
  • Flakiness measurement: flci soak --runs 20 gives per-fixture pass rates, p50/p95 timings, every distinct decode, and the RFC evidence table.
  • Regression gate: flci compare base.xml candidate.xml lists what broke, what got fixed and what lost coverage. CI compares every run against the last green main.
  • Battery logging: each result records both devices' charge level, since low charge shortens RF/RFID range.
  • CI: self-hosted bench workflow, DUT flashing via update install, JUnit check run plus a Markdown job summary.
  • Hardware: parametric OpenSCAD jig (STL rendered in CI) and a wiring guide.

Install

git clone https://github.com/Krishita17/flipper-loopback
cd flipper-loopback
python3 -m venv .venv && source .venv/bin/activate
pip install -e '.[dev]'          # add ,badusb on Linux for HID capture

Run

With no hardware, the parser and orchestration tests run and the hardware tests are skipped with a reason. A skip is never reported as a pass:

pytest

With two Flippers on USB (close qFlipper first, since it holds the serial port):

flci doctor                                   # is the bench ready? what to fix
export FLCI_DUT=<port or device name>         # flashed with the build under test
export FLCI_EMITTER=<port or device name>     # known-good counterpart
export FLCI_CAPABILITIES=subghz,infrared,rfid,nfc   # what your bench is set up for
pytest -m hardware -v                         # JUnit XML -> reports/junit.xml
Variable Meaning
FLCI_DUT, FLCI_EMITTER Required. Port path or Flipper name. Roles are never guessed
FLCI_CAPABILITIES Subsystems this bench supports. Default subghz,infrared,rfid,nfc; add ibutton, gpio once wired, and badusb for operator runs
FLCI_DIRECTIONS dut_rx, dut_tx or both (default both)
FLCI_TAGS Only run fixtures with these tags, e.g. smoke, negative
FLCI_EXCLUDE_TAGS Leave out fixtures with these tags (default environment)
FLCI_OPERATOR=1 Allow the operator-assisted BadUSB test (run with pytest -s)
FLCI_STRICT=1 Treat missing CLI commands and region locks as failures instead of skips

A failing round trip prints what was expected, every packet the DUT decoded, both firmware versions, and the tail of the DUT's raw output.

The flci command

Command What it does
flci doctor [--probe] Check the bench is ready; --probe runs one smoke case per subsystem
flci detect List connected Flippers, firmware versions and assigned roles
flci fixtures List and validate every fixture
flci soak --runs 20 [--subsystem rfid] [--tag smoke] [--direction dut_tx] Repeat cases, write reports/soak.{json,md} with suggested timing budgets
flci report reports/junit.xml [--html out.html] Markdown summary of a run (or --soak reports/soak.json), optional HTML page
flci compare base.xml cand.xml Regressions between two runs; exit code 1 if any
flci record from-sub/from-ir/live ... Create a fixture from a saved file or a live capture
flci flash PACKAGE Flash the DUT from an update package and verify it

Comparing two firmware builds on one bench:

flci flash old-update.tgz && pytest -m hardware --junitxml=reports/base.xml
flci flash new-update.tgz && pytest -m hardware --junitxml=reports/cand.xml
flci compare reports/base.xml reports/cand.xml

Fixtures

Each test case is a YAML file in fixtures/<subsystem>/, sometimes with a stimulus file (.sub, .nfc, BadUSB .txt) next to it. Adding a case doesn't need any code changes:

id: ir_nec_04_08
subsystem: infrared
stimulus: { kind: message, protocol: NEC, address: "0x04", command: "0x08", bursts: 3 }
expected: { subsystem: infrared, protocol: NEC, payload: "04-08" }
tags: [smoke]

You can also record fixtures from real captures or saved Flipper files:

flci record from-sub  my_remote.sub --id gate_433
flci record from-ir   tv.ir --signal Power --id tv_power
flci record live rfid --port "$FLCI_DUT" --id my_fob

Negative cases and timing budgets are fields on the same YAML:

id: rfid_silence
subsystem: rfid
stimulus: { kind: silence, read_timeout_s: 4 }   # send nothing
expect_no_decode: true                           # any decode fails the case
max_duration_s: 8                                # optional timing budget

A broken fixture (bad YAML, unknown subsystem, missing stimulus file, duplicate id, both or neither of expected/expect_no_decode) stops the run with the file name and the reason.

CI on a bench

.github/workflows/hil.yml runs the suite on a self-hosted runner with both Flippers attached, optionally flashes the DUT first (flci flash, stock update install, no DFU), publishes JUnit as a HIL results check, writes a Markdown job summary and an HTML report artifact, and compares against the last green main run to flag regressions. It starts with flci doctor, so a misconfigured bench fails fast with a clear reason. A manual run can also soak every fixture N times. It runs one job at a time (there's only one bench) and never runs fork PRs. Setup is in docs/CI.md.

Hardware

  • hardware/jig/: parametric OpenSCAD jig (back-to-back for RF/RFID/NFC, head-to-head for IR). CI renders the STLs.
  • hardware/WIRING.md: GPIO and iButton jumpers.

Docs

Architecture · Roadmap & nondeterminism log · CI setup · Upstream RFC

Safety

  • Only use hardware and signals you own.
  • Keep TX power low and use shielding or short distances so a bench rig doesn't transmit onto other people's devices or licensed bands. Follow your region's rules. The firmware enforces region locks and the harness treats a refused TX as an error. It never works around one.
  • This project checks decode correctness of firmware. It is not a way to defeat any security system. Fixture keys and IDs are synthetic test values.

Contributing

The most useful thing right now is a bench report: run flci soak --runs 20 and open an issue with the generated reports/soak.md, your firmware versions and jig spacing. New fixtures and CLI fixes are welcome too (see CONTRIBUTING.md).

License

MIT

Releases

Packages

Contributors

Languages