Skip to content

Layer 1 microVM: a php-fpm that refuses to start says why in the launcher's log (#87) - #92

Merged
samifouad merged 9 commits into
mainfrom
claude/microvm-boot-reason-87
Nov 12, 2026
Merged

samifouad merged 9 commits into
mainfrom
claude/microvm-boot-reason-87

Conversation

@samifouad

Copy link
Copy Markdown
Member

Fixes #87: when php-fpm refuses to start in the guest, the launcher's log says why. Stacked on #91 (#86), which it needs: the reason goes over #86's acknowledged log channel. This PR targets claude/microvm-log-86.

Root cause (two parts)

  1. The reason was never written. The guest image's php.ini has error_log = /dev/stderr, and the guest's /dev is a bare devtmpfs. udev makes /dev/stderr, /dev/stdout, /dev/stdin and /dev/fd on a normal system, and the guest has no udev, so those links didn't exist.
    • Every PHP error that isn't a request's went nowhere, the extension's MINIT refusal included. That is why the issue found it neither in the launcher's log nor on the console.
    • Request-time lines still arrived through FPM's fallback logger, as NOTICE: PHP message: ….
  2. Init stopped too fast. When php-fpm's master exited, init returned at once (kernel panic, VM gone), with nothing reported and nothing drained.

What changed

  • The /dev links. securecore-init creates /dev/fd and /dev/std{in,out,err} (pointing into /proc/self/fd) right after mounting devtmpfs.

  • A reported exit. When php-fpm's master exits, init:

    • waits up to 3 s for that master's log pump to read the rest of its output;
    • reports php-fpm exited at boot, before it served (exit code N) (or while serving, or killed by signal N) to the host's log, and php-fpm's last 20 lines plus that line to the console;
    • waits for the host to acknowledge them, then stops.

    A failed boot step is reported the same way. The log sender now starts before anything else in the boot.

  • The launcher's stop line quotes the guest. Each VM's Inbox keeps the guest's last 6 lines. Both "Firecracker exited before php-fpm answered" and "Firecracker exited (while serving)" quote them, instead of only pointing at console.log. When the guest said nothing, the launcher says so.

  • test/ext-vault.sh 5b reads only the launcher's log on the microVM (it no longer appends console.log). It also checks that the launcher's own stop line quotes php-fpm exited at boot … (exit code.

Evidence ([xps]; flock + sudo, logged, exit codes checked; fresh projects sc86-*/sc86m-*)

With this PR (and #91 under it), all eight suites pass on the microVM target:

rust (clippy -D warnings + cargo test, whole workspace)  exit=0
mvm-config      exit=0  39 ok 0 FAIL 12 n/a
mvm-attribution exit=0   6 ok 0 FAIL 34 n/a
mvm-layer3      exit=0  55 ok 0 FAIL  4 n/a
mvm-layer1      exit=0  55 ok 0 FAIL 18 n/a
mvm-hosts       exit=0  37 ok 0 FAIL 59 n/a
mvm-enforce     exit=0  92 ok 0 FAIL 18 n/a
mvm-db          exit=0 152 ok 0 FAIL  0 n/a
mvm-vault       exit=0  70 ok 0 FAIL  9 n/a     (was 68 ok 1 FAIL: this issue)
docker-vault    exit=0  78 ok 0 FAIL
test/ext-microvm.sh log (#86's stress, the /dev/stderr change routes PHP's lines differently)  exit=0 8 ok

The launcher's log for 5b now has the guest's lines and its own stop line, which quotes them:

guest: [11-Nov-2026 19:46:52 UTC] PHP Warning:  PHP Startup: securecore: layer2.vault: SC_LIT_SECRET is a string literal in sc-lit-config.php and opcache inlines it (DECLARE_CONST), so the vault cannot hide it; move it to a vault file (from.vaultFile) or an environment variable (secrets[].env); refusing to start in Unknown on line 0
securecore-fpm: …/securecore.json: the microVM: Firecracker exited (exit status: 0) before php-fpm answered; the guest's last lines: [… PHP Warning: … SC_LIT_SECRET is a string literal … refusing to start …] | [… PHP Fatal error:  Unable to start securecore module …] | securecore-init: php-fpm exited at boot, before it served (exit code …) …
  • Unit tests: a_wait_status_says_the_exit_code_or_the_signal, a_guest_that_stops_at_boot_is_quoted, and Inbox::recent in the_host_logs_each_frame_once_and_counts_what_is_missing.
  • Clippy -D warnings (linux target): clean.

Revert proof

  • Before this PR (Layer 1 microVM: lines the guest's php-fpm logs go missing from the launcher's log #86's code, same suite): FAIL opcache: the log explains the refusal. The launcher's only line was Firecracker exited (exit status: 0) before php-fpm answered; see …/console.log.
  • With init's exit report but without the /dev/stderr link (an intermediate build of this PR), the launcher got guest: securecore-init: php-fpm exited at boot, before it served (exit code 254) but not the extension's own words, and 5b still failed. Both parts are needed.

closes #87

-claude

…r's log (#87)

When php-fpm's master exits, the guest's init reads the rest of its output, reports its exit
(code or signal, at boot or while serving) to the host's log and the console, and waits for the
host to acknowledge those lines before it stops. The log's sender starts first in the boot, so a
failed boot is reported there too. The launcher's stop line quotes the guest's last lines instead
of only pointing at console.log. test/ext-vault.sh 5b checks the reason in the launcher's log
alone.
# Conflicts:
#	ext/fpm/src/guest.rs
#	ext/fpm/src/guestlog.rs
#	ext/fpm/src/vm.rs
…aped and labelled (#87 review)

The launcher's stop lines quoted the guest's last log lines as they were, inside a
securecore-fpm: line, so a guest line could close the quote or start what reads as a launcher
line. Each is now quoted and escaped (Rust's debug escaping: quotes, newlines, control
characters), last in the line, after "the guest's last log lines, as the guest wrote them
(quoted):".
@samifouad

Copy link
Copy Markdown
Member Author

Review fixes (#87 review), in bba3dae (after merging #91's fixes in 0d4897a)

Finding Fix Test (fails with the fix reverted)
3. Guest text sat inside securecore-fpm: lines: exited_before_ready and the "Firecracker exited" line quoted Inbox::recent as it was Each of the guest's lines is now quoted and escaped (Rust's debug escaping: quotes, newlines, control characters). They come last in the launcher's line, after the label the guest's last log lines, as the guest wrote them (quoted): (quoted_guest) the_host_logs_each_frame_once_and_counts_what_is_missing: a guest line x" | securecore-fpm: forged\nsecurecore-fpm: also comes out as one escaped string with no newline. Reverted: left: "l15 | … | x\" | securecore-fpm: forged\nsecurecore-fpm: also". Also a_guest_that_stops_at_boot_is_quoted

ext-vault 5b's check (… before php-fpm answered … php-fpm exited at boot, before it served (exit code) still matches inside the quotes.

Evidence

-claude

@samifouad

Copy link
Copy Markdown
Member Author

Round-3 verification fixes for this PR (K2, S1, K4, K5, N1, K3, N3, the docs and a reload fix) are tabled here: #95 (comment)

-claude

@samifouad
samifouad changed the base branch from claude/microvm-log-86 to main November 12, 2026 00:41
@samifouad
samifouad merged commit 5a1a7d9 into main Nov 12, 2026
3 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Layer 1 microVM: a php-fpm that refuses to start at boot leaves no reason in the launcher's log

1 participant