Repository navigation
subsys: framed byte streams (SPEC-015 §7, OBS-019) - #139
Merged
Merged
Conversation
M4.3's first piece: the framing every byte-stream transport carries, so one line holds the log, the trace, the crash record, the shell and the update protocol. <emb/frame.h>, behind CONFIG_EMB_FRAME (on by default; an image keeps it only if it calls it): emb_frame_encode() writes COBS(channel, payload, CRC-16) and a zero delimiter into the caller's buffer, EMB_FRAME_ENCODED_MAX(n) bounding it; emb_frame_decode_byte() decodes a stream a byte at a time, from a thread or an interrupt, into the caller's buffer, drops a frame with a bad CRC, a channel above 15 or a cut block (errors) or one longer than the buffer (overruns), and resynchronizes at the next zero. API IDs 165 to 167, after the device model's 149 to 164 (#137). subsys/ holds the subsystems over the kernel, built as emb_subsys (an interface library when none is selected), compiled like the kernel and linked into every image. Kconfig gains its Subsystems menu; CI's clang-format step covers subsys/. tools/transport/frame.py is the host's side, written from the specification: encode, decode, a stream decoder, and a tool printing a port's or a capture's frames by channel. The C tests check the encoder against vectors the host tool produces, so the two sides cannot drift apart. A113 (proposed) settles what SPEC-015 §7 left open: channels 0 to 4 are the system's, 4 the update protocol's as SPEC-019 has it, 5 to 15 the application's; the CRC is CRC-16/IBM-3740 (0x1021, initial 0xFFFF; check value 0x29B1), sent least significant byte first. Checked: test_frame's 7 cases (vectors, the 254-byte run boundary, round trips, resync, overrun, misuse, and a fixed-seed stream of 200 frames with one in eight damaged, the same 383 assertions on every target) pass on native (GCC, Clang, tiny, release, ASan), the MPS2 AN385, AN386 (and tiny), AN500 and AN505, the micro:bit, RV32 virt (GCC, Clang), the Uno and the NUCLEO on EmbSIM; the host tool's 7 tests; clang-tidy 18, 0 findings; the docs build. Signed-off-by: teo1747 <motsoujunior17@gmail.com>
teo1747
marked this pull request as draft
November 9, 2026 15:00
CI's lint job: the resource-bound inventory gains frame_bytes, the bytes of one frame, which the caller's buffers bound and the decoder enforces by dropping a longer frame, for emb_frame_encode() and emb_frame_decode_byte(); and the interrupt-safe function table (docs/user/interrupts.md) lists the three framing functions. Every lint step's command passes locally. Signed-off-by: teo1747 <motsoujunior17@gmail.com>
…port-frame Conflicts: CI's tool tests (the framing's host tests and EmbSIM's runner both kept), the conformance list (test_frame, then the device and class tests), the amendments (A113, then A114 to A124), the API registry (main's, then the framing's three entries renumbered 247 to 249, after the configurator's task IDs 241 to 246 on #146, so neither collides whichever merges first), and the bounds inventory (main's and frame_bytes); api_ids.h, api-ids.json, the bounds page and the interrupt-safe table regenerated. Signed-off-by: teo1747 <motsoujunior17@gmail.com>
tools/dev/file_headers.py --write on the files this branch added before the header (CS-11.1) merged: frame.h, frame.c, the subsystems' build and configuration, the conformance test and the host's codec and its tests. Signed-off-by: teo1747 <motsoujunior17@gmail.com>
What parked #139: rule 12.1 (precedence made explicit in every compound condition and the CRC loop's bound), 17.8 (crc16() keeps its argument and works on a local), and cppcheck's variableScope (the decoder's result declared where it is used). What stays is what the tree accepts everywhere for a public API: 11.5 (the payload a const void *), 8.7 (the three public functions) and 2.5 (the channel macros in frame.h), recorded in the baseline. test_frame passes on native Clang. Signed-off-by: teo1747 <motsoujunior17@gmail.com>
The frame's channels (0 log, 1 trace, 2 crash, 3 shell, 4 update, 5 to 15 the application's) and its CRC-16/IBM-3740 over the channel and the payload, low byte first, accepted as written on 2026-11-13. Signed-off-by: teo1747 <motsoujunior17@gmail.com>
The owner chose on 2026-11-13 the ring's critical-section reservation over a lock-free one, both backends with the deferred one the default, and the framed console as an option. A127 writes those choices out: the call sites, the record both backends share, identifiers as offsets in a non-allocated section, the ring's reservation, commit, publication and overflow rules, the log thread and the console's frames. Signed-off-by: teo1747 <motsoujunior17@gmail.com>
The console sends frames from interrupts and faults as well as threads, and a buffer for the encoded frame would be one more thing to share or to put on a small stack. emb_frame_write() reads the payload in place, finding each COBS run's end by looking ahead, and hands the sink a run's bytes in one call. It writes exactly what emb_frame_encode() writes, which the new case checks around the 254-byte run with no zero, zeros inside and a zero last. API id 250. Signed-off-by: teo1747 <motsoujunior17@gmail.com>
Logging (SPEC-015 §2): EMB_LOG_ERR() to EMB_LOG_DBG() with a printf format the compiler checks, and up to eight arguments typed by a signature _Generic computes. A call packs its record in the caller, from any context: a bounded critical section reserves its room in the ring, the copy runs with interrupts enabled, one byte store commits it. The log thread takes records in order and stops at one not committed; a record that does not fit is dropped, counted, and announced by the next one that fits. Two backends share the record: the deferred one sends it and tools/logdec formats it on the host from the ELF, where the call sites' metadata lives in a non-allocated section at address 0 (its first 8 bytes empty, so no metadata sits at the null pointer); the plain-text one formats it in the log thread. The console: every board writes raw bytes with emb_board_write(), and emb_board_puts() is common code. With CONFIG_EMB_CONSOLE_FRAMED everything on it travels in frames, a zero before each: text on channel 3, logs on 0. tools/transport/frame.py keeps bytes that are not a frame when they read as text. examples/logging prints the same lines with both backends on QEMU AN386 and RV32 virt (tools/logdec/backends_agree.py); the deferred image is 3 KB smaller. test_log checks the record's layout, the formats, the levels, the drops and an interrupt's record held back by an uncommitted one. API ids 251 to 255. Signed-off-by: teo1747 <motsoujunior17@gmail.com>
The application job builds examples/logging plain and framed for the native board, the MPS2 AN386 and RV32 virt, runs each, decodes the framed console with tools/logdec and compares the lines with backends_agree.py. The log decoder's own tests run with the other host tools'. A native Linux image is position-independent, so logdec reads a pointer the file holds as zero from its RELATIVE relocation. Signed-off-by: teo1747 <motsoujunior17@gmail.com>
On the table scheduler each priority holds one thread, so a log thread took the test runner's: CONFIG_EMB_LOG_THREAD is off there, and the application sends the records with emb_log_flush(). The plain-text backend's call sites live in program memory on a Harvard core and are read through EMB_FLASH_READ_* (EMB_FLASH_READ_U32 is new); CONFIG_EMB_LOG_FLOAT, off on tiny, leaves out on-target %f. Logging is not offered on AVR in M4.3: the formatter and its stack do not fit the Uno's 2 KB beside a test. size_t is 16 bits on AVR: the conversions are explicit. test_log passed a uint32_t to %u, which is unsigned long on Arm and RISC-V newlib; Doxygen learns EMB_PRINTF_FORMAT. Signed-off-by: teo1747 <motsoujunior17@gmail.com>
emb_trace_start() makes a RAM ring the trace sink: each kernel event becomes a 20-byte record with the cycle counter's time, written in the critical section the event is in. The ring keeps the newest events and numbers them, so a reader knows what was overwritten. emb_trace_dump() sends it on the framed console's channel 1, the threads' names first; CONFIG_EMB_TRACE_STREAM has a thread send new events every period. tools/trace/trace.py extends the counter across its wraps and writes a CTF 1.8 trace and Chrome's JSON for Perfetto; CI checks that Babeltrace 2 reads the trace examples/tracing dumps under QEMU. The tracing example ran on the NUCLEO-F446RE: 256 events over 16 ms at the cycle counter's resolution. Also for the checks: the log and console functions' bounds are in the resource-bound inventory, and every new public function has a fault registry id (256 to 260). Signed-off-by: teo1747 <motsoujunior17@gmail.com>
…dump BLD-005's facade, from per-board defaults: a build directory knows its board, its image and its console (CMakeCache.txt, the board's hw.json), so flash, monitor, hil and trace dump need only -B. flash uses the board's own probe as run_hil.py does. monitor shows a plain console's text as it comes and decodes a framed one by channel: log records with the image's ELF (and --where for file and line), crash records, the shell's text, trace packets kept with --trace-out. trace dump turns the packets emb_trace_dump() sends into CTF and Perfetto's JSON. On the NUCLEO-F446RE: emb build --framed, emb flash and emb monitor showed examples/logging's records decoded, with their call sites. Signed-off-by: teo1747 <motsoujunior17@gmail.com>
The relay passed the other board's TX to the ST-LINK's port only. Now the host's bytes go out of PA9 to the other board too, every byte as it is, each way through a ring of its own so a busy direction never holds up the other: the shell and update channels can reach the board. It also reads the host's stream for frames on channel 15, passed on as well, and drives three bench pins: a reset (PA8, 20 ms low), a power cut (PB5) and a wake-up pulse (PB4). The reset and the power cut are released as inputs so the other board's pull-ups hold its pins. Each command is answered with a channel-15 frame between two of the other board's frames, never cutting one. On the NUCLEO-F446RE: ping and wake answered ok, an unknown command '?'. Signed-off-by: teo1747 <motsoujunior17@gmail.com>
The board reads its console's UART through the UART class, a line or what arrived before the line went quiet, and sends it back in capitals with its length. Behind the NUCLEO's relay it checks the host's way to the Pico 2 W (PA9 to GPIO1), which the shell and update channels use. On the NUCLEO-F446RE's own console: 'hello' came back 'echo 6: HELLO'. Signed-off-by: teo1747 <motsoujunior17@gmail.com>
tools/hil/log-soak logs a numbered record every 10 ms, a burst of 100 every second (more than the ring keeps, so records are dropped and counted) and both rings' counters every 5 s, while the trace streams on channel 1. tools/hil/log_soak.py reads it for a while and passes it when no frame is damaged, every record decodes and each numbered record missing is covered by a dropped record after it. A minute on the NUCLEO-F446RE: 5099 frames, none damaged, 6000 numbered records. test_log gains three threads at three priorities and an interrupt logging at once while the test drains the ring: each producer's records arrive whole and in order, and those kept plus those counted dropped are all that were written. It passes natively, under ASan and TSan, and under QEMU on the Cortex-M4 and the Cortex-M0. The trace stream's thread needed <emb/sched.h>. Signed-off-by: teo1747 <motsoujunior17@gmail.com>
docs/user/logging.md: modules and levels, the two backends, the framed console and emb monitor, recording and viewing the trace in Babeltrace 2 and Perfetto, and the emb tool's commands. The change note lists M4.3's additions. Signed-off-by: teo1747 <motsoujunior17@gmail.com>
So a run can end it: the board's exit, which on the Pico 2 W built for the bench reboots into BOOTSEL for the next image. Signed-off-by: teo1747 <motsoujunior17@gmail.com>
The counter thread logged on the same tick as the bursts, when the ring is full of a burst, so its two records were dropped every time (the soak's accounting showed 39 dropped at each fifth second instead of 37). It now logs half a second later. The change note's heading takes its topic and date, as the lint wants. Signed-off-by: teo1747 <motsoujunior17@gmail.com>
…race calls Regenerated by tools/docs/isr_table.py from the headers' @ctx tags. Signed-off-by: teo1747 <motsoujunior17@gmail.com>
The sink compares each byte with what emb_frame_encode() wrote as it comes, instead of copying the frame into a 320-byte buffer of its own, which took the Uno's static data to 1.25 KiB of its 2 KiB and ran the test into its stack under EmbSIM. Signed-off-by: teo1747 <motsoujunior17@gmail.com>
… cheap to fix clang-tidy: emb_log_format_check_ loses its trailing underscore (the naming rule); the %f formatter's sign was stored twice; the trace packet's slot index widens before it multiplies. The switches over the arguments' types are wrapped in a NOLINT scope for bugprone-branch- clone: their cases differ in the type va_arg reads, which the check does not compare. MISRA: the two rings' statics and the log's put8 take names of their own (5.9: log_ring, log_stats, rec_put8; trace_ring, trace_stats), which also tells them apart in a debugger; to_digits and the %f formatter no longer modify a parameter (17.8); one conditional is parenthesized (12.1). Signed-off-by: teo1747 <motsoujunior17@gmail.com>
Taken from CI's misra-baseline-now at 34dde4f. The log, trace and console code adds what every kernel file baselines: 8.7 on public functions, 2.5 on macros a header offers, 10.x on status and level comparisons, and 17.1 on <stdarg.h>, which the variadic log call needs. Each board's emb_board_write() converts its void pointer once (11.5). Lower: the boards' old puts no longer modify their parameter (17.8), and native and exec lost an unused external function each. Signed-off-by: teo1747 <motsoujunior17@gmail.com>
… variableScope) Signed-off-by: teo1747 <motsoujunior17@gmail.com>
The suites at 34dde4f: 49 of 49 on the NUCLEO-F446RE and 51 of 51 on the Pico 2 W, with GCC and with Clang, test_frame and test_log among them, the producers-at-once case on both. The framed console's soak, 30 minutes on each board: 149,411 and 145,624 frames, none damaged, every numbered record missing counted by a dropped record; a 5-minute soak on the NUCLEO shows the rings' counters. The logging example's backends print the same lines on both boards, the trace example's 256 events decode, and console_echo came back through the relay both ways. The soaks' archives keep the counts, the first 256 KiB of the capture and the whole capture's size and SHA-256. The support matrix follows. Signed-off-by: teo1747 <motsoujunior17@gmail.com>
…transport-frame Both branches added amendment rows (A125 to A128, kept in order), fault registry ids (241 to 246 and 247 to 260, both kept; api_ids.h regenerated), resource bounds, MISRA baseline entries (the higher count where both changed one) and the interrupt table (regenerated). The board build directory is #146's, with the console's library links. The generated linker scripts move to the new layout with the log section; six whose baseline git updated in place of the file were regenerated. emb finds hw.json under generated/docs/. Native 42/42, QEMU Cortex-M4 42/42, Cortex-M0 18/18, Uno 17/17; the logging backends agree under QEMU; the controller, tracing and tasks examples build; emb-hwgen check --strict is clean. Signed-off-by: teo1747 <motsoujunior17@gmail.com>
Signed-off-by: teo1747 <motsoujunior17@gmail.com>
Taken from CI's misra-baseline-now at 6e4f4af. The two generated headers both branches added to (config.h, api_ids.h) count both sides' macros; the counts the merge kept at the higher of the two come down to what the code has. Signed-off-by: teo1747 <motsoujunior17@gmail.com>
teo1747
marked this pull request as ready for review
November 13, 2026 06:23
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
M4.3's first piece: the framing every byte-stream transport carries. One line can then hold the log, the trace, the crash record, the shell and the update protocol. It is independent of the device model (#137) and based on
main.Needs your decision: A113. SPEC-015 §7 gives channels 4 to 15 to the application, while SPEC-019 §6 runs the update protocol on channel 4. §7 also names "a CRC-16" without saying which. A113 proposes:
What it adds
<emb/frame.h>, behindCONFIG_EMB_FRAME. It is on by default, and an image keeps it only if it calls it. Every function is callable from a thread or an interrupt.emb_frame_encode()writesCOBS(channel, payload, CRC-16)and a zero delimiter into the caller's buffer.EMB_FRAME_ENCODED_MAX(n)bounds the size.emb_frame_decoder_init()andemb_frame_decode_byte()form a stream decoder. It drops a frame with a bad CRC, a channel above 15 or a cut block (counted as errors), or one longer than its buffer (counted as overruns). It resynchronizes at the next zero.tools/fault/generate.pyrerun on the merge.subsys/: the subsystems over the kernel, built asemb_subsysand linked into every image. Kconfig gains a Subsystems menu, and CI's clang-format step now coverssubsys/.tools/transport/frame.py: the host's side, written from the specification. It provides encode, decode and a stream decoder, plus a tool that prints a serial port's or a capture's frames by channel. The C tests check the target's encoder against vectors this tool produces, so the two sides cannot drift apart.Checked locally
test_frame's seven cases: vectors, the 254-byte COBS run boundary, round trips, resynchronization, overrun, misuse, and a fixed-seed stream of 200 frames with one in eight damaged. The stream case makes the same 383 assertions on every target.Signed-off-by: teo1747 motsoujunior17@gmail.com