Repository navigation
layer 1 microVM: the guest is the host's own PHP, validated; every check before auto commits (#101) - #102
Conversation
…ted (#101) The launcher captures the host's php-fpm, every extension in its extension folder, their libraries (the loader's own --list), the programs PHP runs and the data they read, and apart from those php.ini, its scan folder and php-fpm's config with its pools. Binaries and libraries must match their package manager's checksums (dpkg, rpm, apk); anything unpackaged must be root's in root-only folders, reached through no link below the top; configs are parsed and refused when they load native code from outside the captured set. Any failure names the file. The PHP layer and the config layer are disks of their own, cached by what they hold; the base (rootfs.ext4) is only the static init, the preload and the warm-up script.
…e database at a bare 3306 (#101) - each php-fpm pool gets a microVM listening where the pool says (listen, listen.owner/group/mode), running as its user, with env[], php_*_value and pm.*; - prepare_all does everything a start does before the boot (capture, pool, sockets, vault env, claim_state, space with a margin, every disk) before the choice, and auto falls back to gelada if the VM fails before it serves; - a bare 3306 reaches the host's MySQL socket (PHP's default socket first) or 127.0.0.1:3306, and the guest has PHP's localhost sockets; - EXTEND_TIMEOUT_USEC while a start prepares; a program the guest lacks is logged; - the capture handles musl's loader, rpm's queries, ldconfig/busybox links to packaged files and /etc configs; test/ext-capture.sh proves dpkg, apk and rpm hosts.
|
Gate status on the final commit (7e6e328), on [xps], under the build lock, each logged to a file with Passed:
Failed for the host environment, not the change:
Not run yet:
Revert proof (only partly done):
-claude |
…M job failures - a pool's listen.allowed_clients and listen.acl_users/acl_groups are enforced on every connection; a bare listen = 9000 binds 127.0.0.1 only; - a pool's own launcher never falls back by itself; the first launcher's wait has a deadline; EXTEND_TIMEOUT_USEC stops once ready, failed or stopping; - a bare 3306 goes where wp-config's DB_HOST says, only to a socket root or the database's user owns in folders only they may write; no /tmp/mysql.sock, no guessed 127.0.0.1:3306; - only this site's pools run as VMs (chdir, doc_root, prefix); another site's refuses the config, so auto falls back once; - data files only when world-readable; values with two quoted strings refused, escapes read as PHP does, pool values passed byte for byte, ffi.preload checked, captured_ro files only; - dpkg diversions followed; rpm SHA-224/SHA-384 checked, unknown algorithms refused; rpm validated before and after it runs; - the installer records the unpackaged files' sha256s (trust on first use) and every capture checks them; - capture.rs unit tests for folders, links, NEVER, the re-hash, setuid, code-loading settings and the record; - CI: the privs probe walks the guest in PHP (no find in the guest), the installer's free-space fixture is half the base image, /etc/hostname and /etc/hosts are captured, and the suites put config files into the php container root-owned.
…he pool sockets as other users - /etc/hostname and the like differ per host: the install's record of unpackaged files covers code only (release-check: the installer's container and the server's differed); - the MySQL-socket test scratches below / (the system temp folder is 1777); - the ACL checks run the ping as uid 33 and 65534 from a folder they can reach; - ext-microvm.sh counts only this site's Firecracker, not another launcher's.
|
The review findings and CI's failures are fixed in 2963be0 and 15fadb9, both pushed onto this branch with no history rewritten. CI run 43182460946: diagnosisThe new validation surfaced all four failures. Each was a test's assumption about the old guest image. release-check:
microVM job:
Running the full gates on [xps] then surfaced one more, in release-check: the install's record of unpackaged files covered Findings and fixes
Revert proofsEach patch removes one check, and then On [xps] (as root, in the toolchain image; the control run without a patch passes 115/115). Every test named below is the only one that failed: On the iMac (the portable checks):
The first [xps] proof run was void: Full gates ([xps], under the build lock, each logged with
|
| gate | result |
|---|---|
docker build --target check (clippy -D warnings, the whole workspace's tests) |
rc=0, fpm 115 passed |
test/ext-microvm.sh |
rc=0, 281 ok, 0 FAIL |
| eight suites, docker target (config, attribution, layer3, layer1, hosts, enforce, db, vault) | 8/8 rc=0; enforce passed on its rerun after config (the first time it ran it found the stack down) |
| eight suites, microVM target | 8/8 rc=0 |
scripts/ext-build.sh 8.3 nts, scripts/guest-build.sh |
rc=0 (guest image 1.9 MB gzipped, 38 MiB unpacked) |
test/release-check.sh 8.3 nts (with the microVM leg) |
rc=0, 104 ok, 0 FAIL |
test/promote-check.sh (zip checked on PATH inside the sudo env) |
rc=0, 14 ok |
test/ext-capture.sh (Debian dpkg, Alpine apk, Fedora rpm) |
rc=0, 24 ok |
Clippy: no warnings.
The first gate run surfaced two test bugs, now fixed:
- the socket-trust test scratched under the system temp folder (1777);
- the ACL pings ran from a folder other users can't read.
Its ext-microvm also failed on a clock-skew check (3 s against the 2 s allowed, a busy box), and it passed on the rerun.
The microVM-target suites ran before 15fadb9, which changes only the record's /etc rule and the tests.
Edits were made with Edit/Write only, with no sed -i and no heredoc edits on repo files.
-claude
…' wait always ends (#102 verification) - a bare 3306: TCP only to MySQL's loopback ports (3306, 33060) or exactly --db-address, connected as the pool user; a unix socket only the database user's, or exactly --db-socket; anything else gives the guest no database and says why; a socket directly in /run, /tmp and the like is bind-mounted alone, never the folder; - wp_db_host skips PHP comments, and two defines that disagree are no answer; - the pools' launchers: SIGTERM, 10 s, SIGKILL, reaped; a stop during the readiness wait is taken at once; --pools-ready-within; a 60 s timeout on every Firecracker API call; - --listen keeps the pool's allowed_clients/ACLs, or refuses when they can't apply; - no record of the unpackaged files: the start refuses (the installer is the only writer).
…real pool launcher The check stopped the command-substitution subshell of the background job (its $(php_root)), not a pool's launcher, so the launcher never started and the test waited forever. The PHP root is now resolved before the job starts, the stopped process is one that runs with --pool, and every wait for a launcher, probe or helper goes through bounded_wait (SIGKILL and a FAIL past its bound). The missing-program probe runs after the clock check (its log wait counted as skew).
…ith no egress handshake release-check found it: connect[].address forwards went through the egress helper, which answers OK first, and that line broke MySQL's protocol. --egress-connect --raw connects and pipes without it.
… the stop lands during the wait
|
This round is a9ecc80 through e378cb0 on this branch, merges only. The 19-hour hangIt was the test, not the launcher. What happened:
The fixes:
Findings and fixes
Revert proofs ([xps], each patch in its own repository): patches 11–18 each fail exactly the test named above. The control run passes. Gates ([xps], under the lock, each under
|
| gate | result |
|---|---|
check target (clippy -D warnings, the whole workspace) |
rc=0, fpm 116 passed |
test/ext-microvm.sh |
rc=0, 303 ok, 0 FAIL |
| eight suites, microVM target | 8/8 rc=0 |
| eight suites, docker target | 8/8 rc=0. ext-enforce reports "stack not up" when it runs right after ext-hosts in this sequence, and passes after ext-config (110 ok). CI doesn't run this docker sequence. |
ext-build.sh, guest-build.sh |
rc=0 |
test/release-check.sh 8.3 nts |
rc=0, 104 ok, 0 FAIL |
test/promote-check.sh |
rc=0, 14 ok |
test/ext-capture.sh (dpkg, apk, rpm) |
rc=0, 24 ok (on a9ecc80; the capture hasn't changed since) |
CI on e378cb0: my one check found all three jobs still pending (run 43384677045).
-claude
Closes #101. Unblocks #100 (the
autoflip). Part of #53.Sami decided on 12 Nov that the microVM adapts to the host. The guest is now built from the host's own PHP, and everything captured is validated.
What changed
The guest is the host's PHP (
ext/fpm/src/capture.rs,pkgdb.rs,phpconf.rs). Each start captures:.soin its extension folder, any.soa config names inside PHP's own folders;ld.so --list, glibc or musl);/bin/sh, sendmail,convert/magick,gs, and thephpCLI that layer 3's helper runs), plus ImageMagick's modules and ghostscript's resources;ld.so.cache,passwd/group(nevershadowor/etc/ssl/private), the CA certificates and OpenSSL's links;The guest has four disks:
rootfs.ext4is now only the SecureCore base: a static init, the preload, the warm-up script and mount points. It is 37 MiB, was 1110 MiB.Validation. Any failure names the file.
microvmthen exits 125;autofalls back to gelada./usris looked up both ways././usrlinks, package-installed links, alternatives;auto_prepend_file,auto_append_fileoropcache.preloadoutside the app root.The root-only manifest is
<state>/capture-manifest.json(0600). It records path, owner, mode, sha256 and source. Changes since the last capture are logged.One VM per pool. Each VM:
listen, with itslisten.owner/listen.group/listen.modeand$poolreplaced (--listenstill wins for one pool);pm/pm.*,env[],php_*_value/php_*_flag,clear_envand the rest;With several pools, the launcher starts a launcher per pool (
--pool <name>, state in<state>/pool-<name>/). It tells systemd once all of them serve, forwards signals, and stops them all if one stops.Every check before
autocommits.prepare_all()runs beforechoose()decides. It covers:env[];claim_state;Also: a VM that
autochose and that fails before it serves drops to gelada (layer 1 isolation auto: gelada, because the microVM failed before it served (…)).microvmstill exits 125.A bare 3306 and
localhost. A{ "port": 3306 }with no socket or address now goes to the host's MySQL socket: PHP'smysqli.default_socket/pdo_mysql.default_socketfirst, then the standard places. Failing that it goes to127.0.0.1:3306, and only then is it egress. The guest gets unix sockets at PHP's default sockets,/run/mysqld/mysqld.sockand/tmp/mysql.sock.A long first start.
EXTEND_TIMEOUT_USEC=30000000is sent every 10 s from preparation until the VM serves, plusSTATUS=lines.A missing program is logged.
guest-init: "convert" isn't in the guest: the host had no such program to capture ….execve/execv/execvpfor direct execs.Smaller pieces:
securecore-fpm --check-captureruns the capture and validation alone.--php-rootpoints at another root (the tests use the php container's/proc/<pid>/root).PROTOCOLis now 6.docs/layer1-microvm.mdhas a new section, "The guest is the host's own PHP (Layer 1 microVM: build the guest from the host's own PHP, validated; fall back before committing #101)".Measurements ([xps], the ext-microvm stack, PHP 8.3.36)
zlib1gfrom trixie-security: capture, PHP layer 1.8 s, snapshot 1.4 s; the site disk was rebuilt too)What was captured on the test stack:
The PHP layer image is 283 MiB (188 MiB on disk).
Evidence
docker build --target check(clippy-D warnings+cargo test, whole workspace) on the first commitcargo clippy -p securecore-fpm --all-targets --target x86_64-unknown-linux-gnu -- -D warningstest/ext-microvm.sh(run 4, with the new checks)all ext-microvm checks passed, 267 ok, 0 FAILtest/ext-capture.sh(new): Debian trixie, Alpine 3.22, Fedora 42all ext-capture checks passedClippy: no warnings.
The new checks in
ext-microvm.sh:/usr/local/sbin/php-fpmwith the host's scan folder;EXTEND_TIMEOUT_USECarrives beforeREADY=1;memory_limit = 201M;microvmexiting 125:DB_HOST=localhostserves WordPress through the host's MySQL socket;pongat its own socket with itslisten.mode, each has its own state, systemd gets oneREADY=1, SIGTERM stops both.ext-capture.shcovers each package manager's own distribution PHP. Each one:opcache.sothat has one byte appended (dpkg, apk and rpm records);extension=/opt/evil/x.so, each naming the file.The suite also found and fixed:
/binlinks were missing;-iprints HTML;--listexits 127 on extensions;%{=TAG};/usr/sbinlink and ldconfig's soname links;/etc/passwd.The status of the remaining gates is in the follow-up comment.
For Sami
sockets.sothat is still root's 0644 passes; only the manifest's change log shows it. A distro PHP (php8.x-fpm from dpkg/apk/rpm) is fully checksum-verified. Pinning the unpackaged files' sha256s at install time, recorded by the installer, would close this. Recommend a follow-up./etc(passwd,group) are ownership-checked only, as configs are.-claude