diff --git a/doc/design/current-limits.md b/doc/design/current-limits.md index 0d94df72c..35d09ebd5 100644 --- a/doc/design/current-limits.md +++ b/doc/design/current-limits.md @@ -18,9 +18,23 @@ The current ABI family intentionally does not provide: - public activation of the reserved control console or an authenticated guest RPC broker; - replaceable sandbox configuration, a production OCI agent, or a public sandbox snapshot workflow; - serialization of live network flows or native host handles; -- snapshotting of the contents of a live virtio-fs export; or +- snapshotting of the contents of a live virtio-fs export; +- more than one virtio-fs share, or different access modes within one share; +- guest creation of symbolic links on the live virtio-fs share; +- caller-owned virtio-fs requests on Windows, or propagation of the guest + caller's supplementary groups; +- mounting a restore-time virtio-fs attachment inside a sandbox workload root; or - compatibility with standalone NVX `MVMSNAP*` or `WHPSNAP*` files. +Sandbox mode does support the live virtio-fs share on a cold boot. The agent +mounts it inside the workload root with `nosuid,nodev`, rejects targets that it +manages or that traverse an image symbolic link, and fails closed if the mount +fails. `--mount-deny` and read-only mode are enforced by the host server as in +the standard profile. With `--mount-owner caller`, Linux hosts perform each +request as the guest caller, map guest root to the export owner, and fail +requests that cannot switch identity with `EPERM`. See +[Live host share in the sandbox](../run.md#live-host-share-in-the-sandbox). + The tier contract and post-restore gate are currently low-level OpenVMM and guest-agent primitives exercised by the native lifecycle suite. The public `nvx sandbox` command does not yet orchestrate platform builds, warm-shim diff --git a/doc/design/sandbox-filesystem-and-agent-architecture.md b/doc/design/sandbox-filesystem-and-agent-architecture.md index abbf18287..5991d13f7 100644 --- a/doc/design/sandbox-filesystem-and-agent-architecture.md +++ b/doc/design/sandbox-filesystem-and-agent-architecture.md @@ -20,11 +20,11 @@ or snapshot-tier metadata alone. ## Implemented filesystem bootstrap The public `nvx sandbox` command accepts one to three role-bearing EROFS lower -images, a preformatted ext4 scratch image, an absolute entrypoint, and -individual argument tokens. It supplies non-secret kernel-command-line -configuration; environment variables, secrets, arguments containing -whitespace, and sandbox snapshot orchestration are not supported by this -command. Lower-level OpenVMM capture and restore do support sandbox blocks. +images, a preformatted ext4 scratch image, an absolute entrypoint, individual +argument tokens, and an optional live virtio-fs share. It supplies non-secret +kernel-command-line configuration; environment variables, secrets, arguments +containing whitespace, and sandbox snapshot orchestration are not supported by +this command. Lower-level OpenVMM capture and restore do support sandbox blocks. See [Run](../run.md#experimental-single-workload-sandbox) and [`scripts/nvx_tools/sandbox.py`](../../scripts/nvx_tools/sandbox.py). @@ -46,10 +46,13 @@ device filter does not mean the current agent installs one. 4. resolve and flush scratch, mount it as ext4 with `rw,nosuid,nodev`, and create its `upper` and `work` directories; 5. mount overlayfs with top-first lower layers and `metacopy=on,xino=on`; -6. start a child behind a FIFO barrier, place that child in the workload +6. when OpenVMM attached a live virtio-fs share, validate its command-line + target, create any missing components without traversing image symbolic + links, and mount the share inside the overlay with `nosuid,nodev`; +7. start a child behind a FIFO barrier, place that child in the workload cgroup before releasing it, and retain a runtime-tmpfs machine-ID file; -7. wait for the child, unmount the overlay, layers, and scratch, and return its - status through the guest exit helper. +8. wait for the child, unmount the share, overlay, layers, and scratch, and + return its status through the guest exit helper. The assembled view is: diff --git a/doc/design/validation.md b/doc/design/validation.md index c756d14dd..03626a0cf 100644 --- a/doc/design/validation.md +++ b/doc/design/validation.md @@ -36,6 +36,13 @@ pre-entry rejection of missing, corrupt, mismatched, or wrong-geometry media. Denied-filesystem coverage verifies listing suppression, allowed writes, direct and parent-relative denial, symlink/junction alias denial, a second virtio-fs mount, and pre-boot rejection of unsafe path policies. +Sandbox live-share coverage boots the real sandbox agent over the Ubuntu EROFS +layer with a read-write share. While the workload runs, it verifies host-file +visibility, denied-path hiding, nested workload writes and `chmod` that appear +on the host immediately, and a host edit that the workload observes. On Linux +it also verifies that caller-owned requests create those files as the export +owner. It then verifies a read-only share that rejects writes, and agent +rejection of reserved and symlinked mount targets before the workload starts. The native suite targets KVM, MSHV, and WHP; a passing run on one backend is not a fresh result for the others. Coverage also includes diff --git a/doc/run.md b/doc/run.md index 79bd66cc2..e137f198b 100644 --- a/doc/run.md +++ b/doc/run.md @@ -252,7 +252,9 @@ backing and are onlined before restore readiness. ## virtio-fs host mapping The microVM reserves one mapping slot with a fixed `microvm` tag. On a cold -boot with `--mount`, the initramfs mounts it automatically: +boot with `--mount`, the initramfs mounts it automatically. In the sandbox +profile, the agent instead mounts it inside the workload root; see +[Live host share in the sandbox](#live-host-share-in-the-sandbox): ```bash python3 scripts/nvx.py run --mount "/mnt/host,/absolute/host/share,rw" @@ -283,6 +285,44 @@ mkdir -p /mnt/host mount -t virtiofs microvm /mnt/host ``` +The share is live host state with no guest caching: entry and attribute cache +lifetimes are zero and file I/O is direct. Host and guest changes are visible to +the other side immediately. In exchange, every path lookup is a round trip to +the host, which metadata-heavy workloads such as package installs will notice. +Files appear exactly as they are on the host, and NVX strips nothing from a live +share. Hide credentials and other sensitive paths with `--mount-deny`. The +device adds no network path, so the network and egress policy are unchanged. + +By default, OpenVMM performs every guest request as its own host identity, so +guest-created files are owned by the OpenVMM user. On Linux, +`--mount-owner caller` performs each request as the guest caller's UID and GID +instead, and maps guest root to the owner of the export root: + +```bash +python3 scripts/nvx.py run --mount "/mnt/host,/absolute/host/share,rw" \ + --mount-owner caller +``` + +Caller mode rejects an export root owned by UID 0 or GID 0. Each request runs +without OpenVMM's supplementary groups and effective capabilities, so a +privileged OpenVMM process cannot lend the guest its privileges. OpenVMM must +run as the export owner or hold `CAP_SETUID` and `CAP_SETGID`. Without those +capabilities, only requests from OpenVMM's own identity, including guest root, +succeed, and they keep OpenVMM's supplementary groups; any other caller gets +`EPERM`. Supplementary groups of the guest caller are not propagated. Because +the guest names the caller, guest root can act as any non-root host user inside +the export. Grant the capabilities only for exports that hold no other users' +files. Windows rejects `--mount-owner caller`: files are created by the OpenVMM +user, and the guest sees attributes derived from that user's access. The owner +mode is host policy rather than snapshot state, so a restore selects it again. + +The microVM has exactly one share, with a single access mode. For example, a +read-write workspace and a read-only tool cache cannot be exported as two +shares. A second share would need a new microVM ABI version. Guest creation of +symbolic links returns `ENOTSUP`, so tools that create links, such as +`npm ci` with `node_modules/.bin`, must run in scratch rather than in the +share. Existing host symbolic links are visible and are resolved by the guest. + ## Experimental single-workload sandbox The `sandbox` command launches the microVM with one to three compressed @@ -336,6 +376,58 @@ The outer agent retains the initramfs root; the capability-stripped child enters only the assembled root with `chroot`, because Linux cannot `pivot_root` away from an initramfs `rootfs`. +### Live host share in the sandbox + +Add `--mount GUEST_TARGET,HOST_PATH[,ro|rw]` to `sandbox run` or +`sandbox provision` to live-share one host directory with the workload: + +```bash +python3 scripts/nvx.py sandbox \ + --layer distro,build/ubuntu-distro.erofs,11111111-1111-1111-1111-111111111111 \ + --scratch /var/lib/nvx/scratch.ext4 \ + --workload-user 1001:1001 \ + --mount /workspace,/home/runner/work/repo,rw \ + --mount-deny /home/runner/work/repo/.secrets +``` + +After it assembles the overlay, and before it starts any workload, the agent +mounts the share at `GUEST_TARGET` inside the workload root with the selected +mode and `nosuid,nodev`. The mount lives below the overlay, so it survives the +workload's private mount namespace and `chroot`. The workload itself still has +no capabilities, so it cannot mount or unmount the share. The share exists only +in the workload root; the agent does not mount it anywhere else. A one-shot +agent unmounts it before the overlay. The agent fails closed with +`NVX-SANDBOX-ERROR` and status 125, never starting the workload, if any of +these holds: + +- the target is not canonical; +- the target is `/etc` or `/etc/machine-id`, or lies under `/proc`, `/sys`, + `/dev`, or `/.nvx-agent`, all of which the agent manages; +- a component of the target in the image is a symbolic link or a + non-directory; +- the mount fails. + +Missing target components are created as root-owned `0755` directories in +scratch. The marker `NVX-SANDBOX-VIRTFS: mounted microvm at GUEST_TARGET (MODE)` +reports a successful mount. Without `--mount`, the sandbox is unchanged. + +The workload accesses the share as its fixed `--workload-user` identity. The +guest kernel checks access against the host owners and modes, so choose the +identity that owns the exported directory, such as the runner user; the image +must define that user, as for any `--workload-user`. On Linux, the command +passes `--mount-owner caller` by default. OpenVMM then performs workload +requests as that identity and agent requests as the export owner, so with an +identity that owns the export, every file the workload creates on the share is +owned by that host user. The export root must not be owned by UID 0 or GID 0. +OpenVMM must run as the export owner or hold `CAP_SETUID` and `CAP_SETGID`; +otherwise the workload's share requests fail with `EPERM`. Use +`--mount-owner process` to perform every request as the OpenVMM process +instead. Windows supports only `process`. A +read-only share rejects workload writes with `EROFS`. `--mount-deny` paths stay +hidden inside the container. The policy, caching, and link limits in +[virtio-fs host mapping](#virtio-fs-host-mapping) apply unchanged. A sandbox +snapshot restore does not mount a newly attached share for the workload. + For a state-aware sandbox, provision configuration without starting a VM, start it once, run multiple workloads in the same warm guest, stop it while retaining configuration, and finally deprovision it: diff --git a/doc/usage.md b/doc/usage.md index c620ea0bb..ac511d846 100644 --- a/doc/usage.md +++ b/doc/usage.md @@ -200,11 +200,15 @@ python3 scripts/nvx.py test-microvm Runs NVX-owned Linux, SMP, virtio, sandbox, and snapshot correctness scenarios against the public OpenVMM CLI. Repeat `--scenario` to select a subset; without it, every scenario supported by the selected guest runs. Alpine remains the -default. Ubuntu rejects the Alpine-control-only `sandbox-blocks` and -`scratch-snapshot` scenarios, the Alpine-prompt-specific `console-snapshot` -scenario, and the sandbox-control-dependent `snapshot-tiers` scenario. The -command requires `build/vmlinux`, the selected initramfs, and -`openvmm/target/release/openvmm[.exe]`. +default. Ubuntu rejects the Alpine-control-only `sandbox-blocks`, +`sandbox-filesystem`, and `scratch-snapshot` scenarios, the +Alpine-prompt-specific `console-snapshot` scenario, and the +sandbox-control-dependent `snapshot-tiers` scenario. The command requires +`build/vmlinux`, the selected initramfs, and +`openvmm/target/release/openvmm[.exe]`. The `sandbox-filesystem` scenario also +requires `build/ubuntu-distro.erofs` and its manifest. On Linux it formats its +scratch image with `mkfs.ext4`; on Windows it copies +`build/ubuntu-smoke-scratch.ext4`. ### `test-adversarial` @@ -296,6 +300,8 @@ python3 scripts/nvx.py run [--memory-capacity-mib MIB] [--processors {1,2,4,8}] [--mount GUEST_TARGET,HOST_PATH[,ro|rw]] + [--mount-deny HOST_PATH]... + [--mount-owner {process,caller}] [--net IPV4/PREFIX] [--network-profile {portable}] [--network-egress {allow,deny}] @@ -322,6 +328,8 @@ python3 scripts/nvx.py run | `--memory-capacity-mib MIB` | none | Reserve an immutable, 128 MiB-aligned RAM capacity for a fresh microVM snapshot. | | `--processors {1,2,4,8}` | `1` | Select the microVM processor count. | | `--mount GUEST_TARGET,HOST_PATH[,ro\|rw]` | none | Expose one host directory to the absolute guest target. Active snapshot restore requires the same canonical path, target, and mode; a dormant-slot restore may attach a new mapping that the resumed guest mounts explicitly. | +| `--mount-deny HOST_PATH` | none | Hide an existing path inside the `--mount` export. Repeat for multiple paths; relative paths are resolved against the export root. | +| `--mount-owner {process,caller}` | OpenVMM default (`process`) | Select the host identity for guest requests on `--mount`. `caller` is Linux-only; see [virtio-fs host mapping](run.md#virtio-fs-host-mapping). | | `--net IPV4/PREFIX` | none | Enable virtio-net with the static guest IPv4 address and prefix. | | `--network-profile {portable}` | none | Select the required cross-platform network behavior contract; must be specified with `--net`. | | `--network-egress {allow,deny}` | `allow` | Set the default guest egress policy. | @@ -354,6 +362,9 @@ python3 scripts/nvx.py sandbox [--hostname NAME] [--memory-max BYTES] [--pids-max COUNT] + [--mount GUEST_TARGET,HOST_PATH[,ro|rw]] + [--mount-deny HOST_PATH]... + [--mount-owner {process,caller}] [--memory-mib MIB] [--hypervisor {auto,whp,kvm,mshv}] [--net IPV4/PREFIX] @@ -371,6 +382,9 @@ python3 scripts/nvx.py sandbox | `--hostname NAME` | `nvx-sandbox` | Set the workload UTS hostname. | | `--memory-max BYTES` | none | Set the workload cgroup memory limit. | | `--pids-max COUNT` | none | Set the workload cgroup process limit. | +| `--mount GUEST_TARGET,HOST_PATH[,ro\|rw]` | none | Live-share one host directory at the absolute target inside the workload root; valid for `run` and `provision`. The target cannot be `/etc`, `/etc/machine-id`, or a path under `/proc`, `/sys`, `/dev`, or `/.nvx-agent`. | +| `--mount-deny HOST_PATH` | none | Hide an existing path inside the export. Repeat for multiple paths; relative paths are resolved against the export root. | +| `--mount-owner {process,caller}` | `caller` on Linux, `process` on Windows | Select the host identity for workload requests on the share. | | `--memory-mib MIB` | `256` | Set guest memory in MiB. | | `--hypervisor {auto,whp,kvm,mshv}` | `auto` | Select the host hypervisor. | | `--net IPV4/PREFIX` | none | Enable virtio-net with a static guest address. | diff --git a/guest/common/init b/guest/common/init index 73dbf906b..05c07e914 100644 --- a/guest/common/init +++ b/guest/common/init @@ -111,6 +111,7 @@ fi mount_virtfs() { /sbin/nvx-hostmount || fatal "virtfs: failed to mount live host directory" } +# The sandbox agent mounts the share inside the container root instead. if [ "$sandbox_mode" = false ]; then mount_virtfs fi diff --git a/guest/common/nvx-init-agent b/guest/common/nvx-init-agent index ebfb047c5..daa7063ad 100644 --- a/guest/common/nvx-init-agent +++ b/guest/common/nvx-init-agent @@ -126,6 +126,60 @@ mount_layer() { fi } +# Mounts the optional live virtio-fs share inside the container root. The +# workload cannot mount it itself because it runs without capabilities, and a +# mount below the overlay survives the private mount namespace and chroot. +mount_virtfs_share() { + virtfs_tag=$(cmdline_value virtfs_tag || true) + [ -n "$virtfs_tag" ] || return 0 + [ "$virtfs_tag" = microvm ] || fatal "unexpected virtio-fs tag: $virtfs_tag" + virtfs_dir=$(cmdline_value virtfs_dir || true) + virtfs_mode=$(cmdline_value virtfs_mode || true) + case "$virtfs_mode" in + ro | rw) ;; + *) fatal "invalid virtio-fs mode: $virtfs_mode" ;; + esac + case "$virtfs_dir" in + /*) ;; + *) fatal "virtio-fs target must be an absolute path: $virtfs_dir" ;; + esac + # The appended slash also catches the root and trailing dot components. + case "$virtfs_dir/" in + *//* | */./* | */../*) + fatal "virtio-fs target is not a canonical path: $virtfs_dir" + ;; + /proc/* | /sys/* | /dev/* | /.nvx-agent/* | /etc/ | /etc/machine-id/) + fatal "virtio-fs target is reserved by the sandbox: $virtfs_dir" + ;; + esac + + # Nothing else runs yet, so checking and creating each component in turn + # cannot race with an image symlink that redirects the mount outside the + # container root. + virtfs_path=$rootfs + virtfs_remainder=${virtfs_dir#/} + while [ -n "$virtfs_remainder" ]; do + virtfs_component=${virtfs_remainder%%/*} + case "$virtfs_remainder" in + */*) virtfs_remainder=${virtfs_remainder#*/} ;; + *) virtfs_remainder= ;; + esac + virtfs_path=$virtfs_path/$virtfs_component + [ ! -L "$virtfs_path" ] || + fatal "virtio-fs target traverses a symbolic link: $virtfs_dir" + if [ ! -e "$virtfs_path" ]; then + mkdir -m 0755 "$virtfs_path" || + fatal "failed to create virtio-fs target: $virtfs_dir" + fi + [ -d "$virtfs_path" ] && [ ! -L "$virtfs_path" ] || + fatal "virtio-fs target is not a directory: $virtfs_dir" + done + mount -t virtiofs -o "$virtfs_mode,nosuid,nodev" "$virtfs_tag" "$virtfs_path" || + fatal "failed to mount the virtio-fs share at $virtfs_dir" + virtfs_target=$virtfs_path + echo "NVX-SANDBOX-VIRTFS: mounted $virtfs_tag at $virtfs_dir ($virtfs_mode)" +} + mkdir -p "$layers" "$scratch" "$rootfs" mount -t tmpfs -o mode=755,nosuid,nodev tmpfs /run || fatal "failed to mount runtime tmpfs" @@ -208,6 +262,8 @@ mkdir -p "$scratch/upper" "$scratch/work" mount -t overlay overlay \ -o "lowerdir=$lowerdirs,upperdir=$scratch/upper,workdir=$scratch/work,metacopy=on,xino=on" \ "$rootfs" || fatal "failed to assemble the container overlay" +virtfs_target= +mount_virtfs_share entrypoint=$(cmdline_value nvx_entrypoint || true) [ -n "$entrypoint" ] || entrypoint=/bin/sh @@ -291,6 +347,10 @@ echo "NVX-SANDBOX-READY: pid=$container_pid layers=$layer_count" status=0 wait "$container_pid" || status=$? rm -f "$runtime/container.pid" "$runtime/workload-machine-id" +if [ -n "$virtfs_target" ] && ! umount "$virtfs_target"; then + echo "NVX-SANDBOX-ERROR: failed to unmount the virtio-fs share" >&2 + status=1 +fi if ! umount "$rootfs"; then echo "NVX-SANDBOX-ERROR: failed to unmount the container overlay" >&2 status=1 diff --git a/openvmm b/openvmm index c9f659c36..65cc1c2a5 160000 --- a/openvmm +++ b/openvmm @@ -1 +1 @@ -Subproject commit c9f659c36c8cb36d5aac745f7c26808296689f3f +Subproject commit 65cc1c2a5a878eb5a0e65eef313c42de301ab19f diff --git a/scripts/nvx.py b/scripts/nvx.py index 2c777da2c..58a5b7cca 100644 --- a/scripts/nvx.py +++ b/scripts/nvx.py @@ -75,7 +75,13 @@ package_release, verify_source_tree, ) -from nvx_tools.sandbox import SandboxLaunch, SandboxLayer, parse_workload_identity +from nvx_tools.sandbox import ( + MOUNT_OWNERS, + SandboxLaunch, + SandboxLayer, + SandboxMount, + parse_workload_identity, +) DEFAULT_RELEASE_REPOSITORY = "microsoft/nvx" HYPERVISORS = ("auto", "whp", "kvm", "mshv") @@ -334,8 +340,12 @@ def command_run(args: argparse.Namespace) -> None: if args.mount.count(",") not in (1, 2): raise ScriptError("--mount must be GUEST_TARGET,HOST_PATH[,ro|rw]") command.extend(["--mount", args.mount]) + elif args.mount_owner is not None: + raise ScriptError("--mount-owner requires --mount") for denied_path in args.mount_deny: command.extend(["--mount-deny", str(denied_path)]) + if args.mount_owner is not None: + command.extend(["--mount-owner", args.mount_owner]) if args.net is not None: command.extend(["--net", args.net, "--network-profile", args.network_profile]) if args.network_egress is not None: @@ -373,11 +383,30 @@ def command_sandbox(args: argparse.Namespace) -> None: raise ScriptError( "--outcome-report is only valid for one-shot run or managed exec" ) + mount_requested = ( + args.mount is not None or bool(args.mount_deny) or args.mount_owner is not None + ) + if mount_requested and operation not in ("run", "provision"): + raise ScriptError( + "--mount, --mount-deny, and --mount-owner are only valid for " + "sandbox run or provision" + ) + if args.mount is None and mount_requested: + raise ScriptError("--mount-deny and --mount-owner require --mount") if operation in ("run", "provision"): if (args.net is None) != (args.network_profile is None): raise ScriptError("--net and --network-profile must be specified together") if not args.layer or args.scratch is None: raise ScriptError(f"sandbox {operation} requires --layer and --scratch") + mount = ( + None + if args.mount is None + else SandboxMount.parse( + args.mount, + denied_paths=tuple(args.mount_deny), + owner=args.mount_owner, + ) + ) launch = SandboxLaunch( layers=tuple(args.layer), scratch=args.scratch, @@ -387,6 +416,7 @@ def command_sandbox(args: argparse.Namespace) -> None: workload_identity=args.workload_user, memory_max=args.memory_max, pids_max=args.pids_max, + mount=mount, ).validated() _validate_sandbox_systemd_policy(launch) else: @@ -718,6 +748,11 @@ def parse_args(argv: Sequence[str] | None = None) -> argparse.Namespace: run.add_argument("--processors", type=int, choices=(1, 2, 4, 8), default=1) run.add_argument("--mount", help="GUEST_TARGET,HOST_PATH,ro|rw") run.add_argument("--mount-deny", action="append", type=Path, default=[]) + run.add_argument( + "--mount-owner", + choices=MOUNT_OWNERS, + help="host identity for guest --mount requests (default: OpenVMM's process)", + ) run.add_argument("--net", metavar="IPV4/PREFIX") run.add_argument("--network-profile", choices=NETWORK_PROFILES) run.add_argument("--network-egress", choices=("allow", "deny")) @@ -784,6 +819,27 @@ def sandbox_identity(value: str) -> tuple[int, int]: ) sandbox.add_argument("--memory-max", type=int) sandbox.add_argument("--pids-max", type=int) + sandbox.add_argument( + "--mount", + metavar="GUEST_TARGET,HOST_PATH[,ro|rw]", + help="live-share one host directory inside the workload root", + ) + sandbox.add_argument( + "--mount-deny", + action="append", + type=Path, + default=[], + metavar="HOST_PATH", + help="hide an existing path inside the --mount export; repeatable", + ) + sandbox.add_argument( + "--mount-owner", + choices=MOUNT_OWNERS, + help=( + "host identity for workload --mount requests " + "(default: caller on Linux, process on Windows)" + ), + ) sandbox.add_argument("--memory-mib", type=int, default=256) sandbox.add_argument( "--timeout", diff --git a/scripts/nvx_tools/microvm_test_scripts/sandbox-filesystem-read-only.sh.in b/scripts/nvx_tools/microvm_test_scripts/sandbox-filesystem-read-only.sh.in new file mode 100644 index 000000000..34ced9cbd --- /dev/null +++ b/scripts/nvx_tools/microvm_test_scripts/sandbox-filesystem-read-only.sh.in @@ -0,0 +1,27 @@ +set -eu +fail() { + code="$1" + echo "NVX-SANDBOX-FILESYSTEM-READ-ONLY-FAIL code=$code" + exit "$code" +} + +share=@TARGET@ +options=$(awk -v share="$share" '$2 == share && $3 == "virtiofs" { print $4 }' /proc/mounts) +case ",$options," in *,ro,*) ;; *) fail 60 ;; esac +case ",$options," in *,nosuid,*) ;; *) fail 61 ;; esac +case ",$options," in *,nodev,*) ;; *) fail 62 ;; esac +[ "$(cat "$share/host-seed")" = NVX-HOST-SEED ] || fail 63 +if ls -A "$share" | grep -qx secrets; then + fail 64 +fi +if cat "$share/secrets/token" >/dev/null 2>&1; then + fail 65 +fi +if error=$(touch "$share/mutation" 2>&1); then + fail 66 +fi +case "$error" in *"Read-only file system"*) ;; *) fail 67 ;; esac +if mkdir "$share/mutation-dir" 2>/dev/null; then + fail 68 +fi +echo NVX-SANDBOX-FILESYSTEM-READ-ONLY-OK diff --git a/scripts/nvx_tools/microvm_test_scripts/sandbox-filesystem.sh.in b/scripts/nvx_tools/microvm_test_scripts/sandbox-filesystem.sh.in new file mode 100644 index 000000000..c40452c8a --- /dev/null +++ b/scripts/nvx_tools/microvm_test_scripts/sandbox-filesystem.sh.in @@ -0,0 +1,33 @@ +set -eu +fail() { + code="$1" + echo "NVX-SANDBOX-FILESYSTEM-FAIL code=$code" + exit "$code" +} + +share=@TARGET@ +options=$(awk -v share="$share" '$2 == share && $3 == "virtiofs" { print $4 }' /proc/mounts) +case ",$options," in *,rw,*) ;; *) fail 40 ;; esac +case ",$options," in *,nosuid,*) ;; *) fail 41 ;; esac +case ",$options," in *,nodev,*) ;; *) fail 42 ;; esac +[ "$(id -u):$(id -g)" = @IDENTITY@ ] || fail 43 +[ "$(cat "$share/host-seed")" = NVX-HOST-SEED ] || fail 44 +if ls -A "$share" | grep -qx secrets; then + fail 45 +fi +if cat "$share/secrets/token" >/dev/null 2>&1; then + fail 46 +fi +mkdir "$share/guest-dir" || fail 47 +printf 'NVX-GUEST-NESTED\n' >"$share/guest-dir/nested" || fail 48 +printf 'NVX-GUEST-WRITE\n' >"$share/guest-file" || fail 49 +@OWNER_CHECK@ +echo NVX-SANDBOX-FILESYSTEM-WRITTEN +tries=0 +while [ ! -e "$share/host-edit" ]; do + [ "$tries" -lt 1200 ] || fail 51 + sleep 0.05 + tries=$((tries + 1)) +done +[ "$(cat "$share/host-edit")" = NVX-HOST-EDIT ] || fail 52 +echo NVX-SANDBOX-FILESYSTEM-OK diff --git a/scripts/nvx_tools/microvm_tests.py b/scripts/nvx_tools/microvm_tests.py index b018d1214..6c126c105 100644 --- a/scripts/nvx_tools/microvm_tests.py +++ b/scripts/nvx_tools/microvm_tests.py @@ -7,8 +7,11 @@ import os import queue import secrets +import shutil import socket +import stat import subprocess +import sys import tempfile import threading import time @@ -40,6 +43,7 @@ from .build_constants import ( BuildConstants, KernelBuildConstants, + UbuntuBuildConstants, ) from .ci import OPENVMM_TEST_BACKENDS, validate_openvmm_test_backend from .common import ( @@ -47,11 +51,20 @@ artifact_path, openvmm_binary_path, require_file, + require_tool, + run_checked, sha256_file, ) from .control_session import ControlSession from .guests import GUEST_NAMES, GuestDescriptor, guest_descriptor from .openvmm_process import OpenvmmProcess, TcpConsole +from .sandbox import ( + DEFAULT_WORKLOAD_IDENTITY, + SandboxLaunch, + SandboxLayer, + SandboxMount, + default_mount_owner, +) MICROVM_TEST_SCENARIOS = ( "console-exit", @@ -71,6 +84,7 @@ "restore-processors", "restore-tsc-sync", "sandbox-blocks", + "sandbox-filesystem", "scratch-snapshot", "smp", "smp-lapic", @@ -82,7 +96,13 @@ "workload-identity", ) UBUNTU_UNSUPPORTED_SCENARIOS = frozenset( - ("console-snapshot", "sandbox-blocks", "scratch-snapshot", "snapshot-tiers") + ( + "console-snapshot", + "sandbox-blocks", + "sandbox-filesystem", + "scratch-snapshot", + "snapshot-tiers", + ) ) MICROVM_PROCESSOR_COUNTS = (1, 2, 4, 8) MICROVM_TEST_SCRIPTS_DIR = Path(__file__).with_name("microvm_test_scripts") @@ -105,6 +125,16 @@ HOST_LOOPBACK_INGRESS_READY_MARKER = b"NVX-HOST-LOOPBACK-INGRESS-READY" HOST_LOOPBACK_PORT_BIND_ATTEMPTS = 16 SANDBOX_BLOCK_SIZE = 8 * 1024 * 1024 +SANDBOX_FILESYSTEM_TARGET = "/workspace" +SANDBOX_FILESYSTEM_WRITTEN_MARKER = b"NVX-SANDBOX-FILESYSTEM-WRITTEN" +SANDBOX_FILESYSTEM_OK_MARKER = b"NVX-SANDBOX-FILESYSTEM-OK" +SANDBOX_FILESYSTEM_READ_ONLY_MARKER = b"NVX-SANDBOX-FILESYSTEM-READ-ONLY-OK" +SANDBOX_FILESYSTEM_EXIT_MARKER = b"NVX-SANDBOX-EXIT: status=0" +SANDBOX_FILESYSTEM_USER = "nvx-share" +# A root test run cannot use UID 0 as the workload, so it exports this owner. +SANDBOX_FILESYSTEM_ROOT_RUN_IDENTITY = (12345, 12345) +SANDBOX_FILESYSTEM_SCRATCH_TEMPLATE_NAME = "ubuntu-smoke-scratch.ext4" +SANDBOX_FILESYSTEM_SCRATCH_SIZE = 64 * 1024 * 1024 SNAPSHOT_CORE_CONTINUED_MARKER = b"NVX-SNAPSHOT-CORE-CONTINUED" SNAPSHOT_CORE_COMPLETION_MARKER = b"NVX-SNAPSHOT-CORE-OK" CONSOLE_BINARY_MARKER = b"\0\r\n\x7f\xffNVX-CONSOLE-BINARY" @@ -1952,6 +1982,323 @@ def run_sandbox_blocks( ) +def _sandbox_filesystem_identity() -> tuple[tuple[int, int], str]: + """Returns the workload identity and mount owner for the live-share test. + + Linux runs use caller ownership with a workload identity that owns the + export, which maps guest-created files to that owner. Windows has no host + UID mapping and keeps the default identity and process ownership. + """ + if sys.platform == "win32": + return DEFAULT_WORKLOAD_IDENTITY, default_mount_owner() + uid, gid = os.getuid(), os.getgid() + if uid == 0: + return SANDBOX_FILESYSTEM_ROOT_RUN_IDENTITY, "caller" + if gid == 0 or 65534 in (uid, gid): + raise ScriptError( + "sandbox-filesystem requires a host UID and GID other than 0 and 65534" + ) + return (uid, gid), "caller" + + +def _assign_sandbox_export_owner(export: Path, identity: tuple[int, int]) -> None: + if sys.platform == "win32": + return + # This also replaces a group inherited from a set-group-ID parent. + for path in (export, *export.rglob("*")): + os.chown(path, *identity, follow_symlinks=False) + os.chmod(export, 0o700) + + +def _sandbox_filesystem_scratch(root: Path, identity: tuple[int, int]) -> Path: + """Creates the scratch template whose upper layer defines the identity.""" + template = root / "scratch-template.ext4" + if sys.platform == "win32": + shutil.copyfile( + require_file( + artifact_path(SANDBOX_FILESYSTEM_SCRATCH_TEMPLATE_NAME), + "Ubuntu sandbox scratch template", + ), + template, + ) + return template + uid, gid = identity + staging = root / "scratch-staging" + upper = staging / "upper" + etc = upper / "etc" + home = upper / "home" / SANDBOX_FILESYSTEM_USER + etc.mkdir(parents=True) + home.mkdir(parents=True) + (etc / "passwd").write_bytes( + ( + "root:x:0:0:root:/root:/bin/sh\n" + "nobody:x:65534:65534:nobody:/nonexistent:/usr/sbin/nologin\n" + f"{SANDBOX_FILESYSTEM_USER}:x:{uid}:{gid}::" + f"/home/{SANDBOX_FILESYSTEM_USER}:/bin/sh\n" + ).encode() + ) + (etc / "group").write_bytes( + f"root:x:0:\nnogroup:x:65534:\n{SANDBOX_FILESYSTEM_USER}:x:{gid}:\n".encode() + ) + for directory in (staging, upper, etc, home.parent, home): + os.chmod(directory, 0o755) + os.chown(home, uid, gid) + with template.open("wb") as image: + image.truncate(SANDBOX_FILESYSTEM_SCRATCH_SIZE) + run_checked( + [ + require_tool("mkfs.ext4", "sandbox-filesystem requires mkfs.ext4 on Linux"), + "-q", + "-F", + "-m", + "0", + "-E", + "lazy_itable_init=0,lazy_journal_init=0", + "-d", + staging, + template, + ] + ) + return template + + +def _sandbox_filesystem_command( + executable: Path, + kernel: Path, + initrd: Path, + backend: str, + memory_mib: int, + launch: SandboxLaunch, + *, + extra_arguments: Sequence[str] = (), + report: Path | None = None, +) -> list[str]: + command = [ + str(executable), + *launch.validated().openvmm_arguments(), + *extra_arguments, + "--microvm-lifecycle", + "one-shot", + "--single-process", + "--hypervisor", + backend, + "--memory", + f"{memory_mib}M", + "--kernel", + str(kernel), + "--initrd", + str(initrd), + "--cmdline", + launch.kernel_command_line("quiet loglevel=0"), + ] + if report is not None: + command.extend(("--microvm-report", str(report))) + return command + + +def _check_sandbox_filesystem_writes(export: Path, identity: tuple[int, int]) -> None: + written = export / "guest-file" + nested = export / "guest-dir" / "nested" + if ( + written.read_bytes() != b"NVX-GUEST-WRITE\n" + or nested.read_bytes() != b"NVX-GUEST-NESTED\n" + ): + raise RuntimeError("sandbox workload writes were not visible on the host") + if sys.platform == "win32": + return + for path in (written, nested.parent, nested): + status = path.stat() + if (status.st_uid, status.st_gid) != identity: + raise RuntimeError( + f"sandbox workload created {path.name} as " + f"{status.st_uid}:{status.st_gid}, not the export owner " + f"{identity[0]}:{identity[1]}" + ) + if stat.S_IMODE(written.stat().st_mode) != 0o600: + raise RuntimeError("sandbox workload chmod did not reach the host file") + + +def run_sandbox_filesystem( + executable: Path, + kernel: Path, + initrd: Path, + backend: str, + *, + memory_mib: int, + timeout: float, + output_dir: Path, +) -> None: + layer_path = require_file( + artifact_path(UbuntuBuildConstants.DISTRO_NAME), + "Ubuntu EROFS distro layer", + ) + manifest_path = require_file( + artifact_path(UbuntuBuildConstants.DISTRO_MANIFEST_NAME), + "Ubuntu EROFS distro manifest", + ) + try: + manifest = json.loads(manifest_path.read_text(encoding="utf-8")) + layer = SandboxLayer( + role="distro", + path=layer_path, + uuid=str(uuid.UUID(str(manifest["uuid"]))), + ) + except (OSError, KeyError, TypeError, ValueError) as error: + raise ScriptError( + f"invalid Ubuntu EROFS distro manifest: {manifest_path}" + ) from error + memory_mib = max(memory_mib, 256) + target = SANDBOX_FILESYSTEM_TARGET + identity, owner = _sandbox_filesystem_identity() + with tempfile.TemporaryDirectory(prefix="nvx-sandbox-filesystem-") as temporary: + root = Path(temporary) + export = root / "export" + secrets_path = export / "secrets" + secrets_path.mkdir(parents=True) + secret = secrets_path / "token" + secret.write_bytes(b"NVX-SECRET\n") + (export / "host-seed").write_bytes(b"NVX-HOST-SEED\n") + (export / "probe.sh").write_bytes( + _render_script( + "sandbox-filesystem.sh.in", + TARGET=target, + IDENTITY=f"{identity[0]}:{identity[1]}", + # Only a caller-owned share reports the workload as the owner + # that the guest kernel requires for chmod. + OWNER_CHECK=( + 'chmod 0600 "$share/guest-file" || fail 50' + if owner == "caller" + else ":" + ), + ).encode() + ) + (export / "probe-read-only.sh").write_bytes( + _render_script("sandbox-filesystem-read-only.sh.in", TARGET=target).encode() + ) + _assign_sandbox_export_owner(export, identity) + template = _sandbox_filesystem_scratch(root, identity) + + def sandbox(name: str, access: str | None, probe: str) -> SandboxLaunch: + scratch = root / f"{name}-scratch.ext4" + shutil.copyfile(template, scratch) + return SandboxLaunch( + layers=(layer,), + scratch=scratch, + entrypoint="/bin/sh", + args=(f"{target}/{probe}",), + workload_identity=identity, + mount=( + None + if access is None + else SandboxMount( + guest_target=target, + host_path=export, + access=access, + denied_paths=(secrets_path,), + owner=owner, + ) + ), + ) + + report_path = root / "sandbox-filesystem.json" + with OpenvmmProcess( + _sandbox_filesystem_command( + executable, + kernel, + initrd, + backend, + memory_mib, + sandbox("read-write", "rw", "probe.sh"), + report=report_path, + ), + output_dir / "sandbox-filesystem.log", + ) as process: + process.wait_for_line( + f"NVX-SANDBOX-VIRTFS: mounted microvm at {target} (rw)".encode(), + timeout, + ) + process.wait_for_line(SANDBOX_FILESYSTEM_WRITTEN_MARKER, timeout) + # The workload is still running, so these checks observe the live + # share rather than a copy-back after exit. + _check_sandbox_filesystem_writes(export, identity) + staged_edit = export / ".host-edit" + staged_edit.write_bytes(b"NVX-HOST-EDIT\n") + os.replace(staged_edit, export / "host-edit") + process.wait_for_line(SANDBOX_FILESYSTEM_OK_MARKER, timeout) + result = process.wait(timeout) + if ( + result.returncode != 0 + or SANDBOX_FILESYSTEM_EXIT_MARKER not in _output_lines(result.output) + ): + raise RuntimeError("sandbox live-share workload did not exit cleanly") + if secret.read_bytes() != b"NVX-SECRET\n": + raise RuntimeError("sandbox workload modified a denied path") + report = _read_outcome_report(report_path) + _preserve_outcome_report(output_dir / "sandbox-filesystem.json", report) + if report["outcome"]["category"] != "success" or not all( + report["teardown"].values() + ): + raise RuntimeError("sandbox live-share outcome reported an unclean exit") + + with OpenvmmProcess( + _sandbox_filesystem_command( + executable, + kernel, + initrd, + backend, + memory_mib, + sandbox("read-only", "ro", "probe-read-only.sh"), + ), + output_dir / "sandbox-filesystem-read-only.log", + ) as process: + result = process.wait(timeout) + lines = _output_lines(result.output) + if ( + result.returncode != 0 + or f"NVX-SANDBOX-VIRTFS: mounted microvm at {target} (ro)".encode() + not in lines + or SANDBOX_FILESYSTEM_READ_ONLY_MARKER not in lines + ): + raise RuntimeError("read-only sandbox share did not reject workload writes") + if (export / "mutation").exists() or (export / "mutation-dir").exists(): + raise RuntimeError("read-only sandbox share was modified") + + for name, rejected_target, expected in ( + ( + "reserved", + "/dev/nvx-share", + b"virtio-fs target is reserved by the sandbox", + ), + ( + "symlink", + "/bin/nvx-share", + b"virtio-fs target traverses a symbolic link", + ), + ): + with OpenvmmProcess( + _sandbox_filesystem_command( + executable, + kernel, + initrd, + backend, + memory_mib, + sandbox(name, None, "probe-read-only.sh"), + extra_arguments=("--mount", f"{rejected_target},{export},ro"), + ), + output_dir / f"sandbox-filesystem-{name}.log", + ) as process: + result = process.wait(timeout) + if ( + result.returncode == 0 + or expected not in result.output + or b"NVX-SANDBOX-READY" in result.output + ): + raise RuntimeError( + f"sandbox agent did not reject the {name} share target " + f"{rejected_target}" + ) + + def run_smp_snapshot( executable: Path, kernel: Path, @@ -3826,6 +4173,17 @@ def run(args: argparse.Namespace) -> int: timeout=args.timeout, log_path=output_dir / "sandbox-blocks.log", ) + if "sandbox-filesystem" in scenarios: + print(f"Running microVM sandbox live filesystem on OpenVMM/{args.backend}") + run_sandbox_filesystem( + executable, + kernel, + initrd, + args.backend, + memory_mib=args.memory_mib, + timeout=args.timeout, + output_dir=output_dir, + ) if "scratch-snapshot" in scenarios: print(f"Running microVM scratch snapshot correctness on OpenVMM/{args.backend}") run_scratch_snapshot( diff --git a/scripts/nvx_tools/sandbox.py b/scripts/nvx_tools/sandbox.py index cd64fadfc..e8fce03b8 100644 --- a/scripts/nvx_tools/sandbox.py +++ b/scripts/nvx_tools/sandbox.py @@ -4,7 +4,7 @@ import os import re -from dataclasses import dataclass +from dataclasses import dataclass, replace from pathlib import Path from uuid import UUID @@ -24,6 +24,18 @@ ) _HOSTNAME = re.compile(r"(?=^.{1,63}$)(?!-)[a-z0-9](?:[a-z0-9-]{0,61}[a-z0-9])?$") DEFAULT_WORKLOAD_IDENTITY = (65534, 65534) +MOUNT_ACCESS_MODES = ("ro", "rw") +MOUNT_OWNERS = ("process", "caller") +MOUNT_DENIED_PATHS_MAX = 128 +# Targets that the sandbox agent itself mounts or rewrites inside the container. +_RESERVED_MOUNT_TREES = ("/proc", "/sys", "/dev", "/.nvx-agent") +_RESERVED_MOUNT_TARGETS = ("/etc", "/etc/machine-id") +_INVALID_MOUNT_TARGET_CHARACTERS = frozenset("\0\\=,") + + +def default_mount_owner() -> str: + """Returns the host identity used for sandbox mounts on this host.""" + return "process" if os.name == "nt" else "caller" def parse_workload_identity(value: str) -> tuple[int, int]: @@ -66,6 +78,108 @@ def parse(cls, value: str) -> SandboxLayer: return cls(role=role, path=Path(raw_path), uuid=uuid) +@dataclass(frozen=True) +class SandboxMount: + """Live host directory mounted inside the sandbox workload root.""" + + guest_target: str + host_path: Path + access: str = "ro" + denied_paths: tuple[Path, ...] = () + owner: str = "process" + + @classmethod + def parse( + cls, + value: str, + *, + denied_paths: tuple[Path, ...] = (), + owner: str | None = None, + ) -> SandboxMount: + fields = value.split(",") + if len(fields) not in (2, 3): + raise ScriptError("--mount must be GUEST_TARGET,HOST_PATH[,ro|rw]") + guest_target, raw_path = fields[:2] + if not raw_path: + raise ScriptError("--mount host path is empty") + return cls( + guest_target=guest_target, + host_path=Path(raw_path), + access=fields[2] if len(fields) == 3 else "ro", + denied_paths=denied_paths, + owner=default_mount_owner() if owner is None else owner, + ) + + def __post_init__(self) -> None: + target = self.guest_target + components = target.split("/")[1:] + if ( + not target.startswith("/") + or target == "/" + or any(component in ("", ".", "..") for component in components) + or any(character.isspace() for character in target) + or not _INVALID_MOUNT_TARGET_CHARACTERS.isdisjoint(target) + ): + raise ScriptError( + "--mount guest target must be a canonical absolute path without " + f"whitespace, backslashes, commas, or '=': {target!r}" + ) + if target in _RESERVED_MOUNT_TARGETS or any( + target == tree or target.startswith(f"{tree}/") + for tree in _RESERVED_MOUNT_TREES + ): + raise ScriptError( + f"--mount guest target is reserved by the sandbox: {target}" + ) + if self.access not in MOUNT_ACCESS_MODES: + raise ScriptError(f"--mount mode must be ro or rw, not {self.access!r}") + if self.owner not in MOUNT_OWNERS: + raise ScriptError(f"unsupported --mount-owner {self.owner!r}") + if self.owner == "caller" and os.name == "nt": + raise ScriptError("--mount-owner caller requires a Linux host") + if len(self.denied_paths) > MOUNT_DENIED_PATHS_MAX: + raise ScriptError( + f"--mount-deny accepts at most {MOUNT_DENIED_PATHS_MAX} paths" + ) + if "," in os.fspath(self.host_path): + raise ScriptError( + f"--mount host paths containing commas are unsupported: {self.host_path}" + ) + + def validated(self) -> SandboxMount: + if not self.host_path.is_dir(): + raise ScriptError( + f"--mount host directory does not exist: {self.host_path}" + ) + for denied in self.denied_paths: + # OpenVMM resolves relative denied paths against the export root. + candidate = denied if denied.is_absolute() else self.host_path / denied + if not os.path.lexists(candidate): + raise ScriptError(f"--mount-deny path does not exist: {candidate}") + return self + + def resolved(self) -> SandboxMount: + """Returns the mount with an absolute export root for later launches.""" + return replace(self, host_path=self.host_path.resolve()) + + def openvmm_arguments(self) -> list[str]: + arguments = [ + "--mount", + f"{self.guest_target},{os.fspath(self.host_path)},{self.access}", + ] + for denied in self.denied_paths: + arguments.extend(("--mount-deny", os.fspath(denied))) + arguments.extend(("--mount-owner", self.owner)) + return arguments + + def command_line_fragment(self) -> str: + """Returns the bootstrap tokens that OpenVMM appends for this mount.""" + return ( + f"virtfs_dir={self.guest_target} virtfs_tag=microvm " + f"virtfs_mode={self.access}" + ) + + @dataclass(frozen=True) class SandboxLaunch: layers: tuple[SandboxLayer, ...] @@ -76,6 +190,7 @@ class SandboxLaunch: workload_identity: tuple[int, int] = DEFAULT_WORKLOAD_IDENTITY memory_max: int | None = None pids_max: int | None = None + mount: SandboxMount | None = None def __post_init__(self) -> None: if not 1 <= len(self.layers) <= len(LAYER_ROLES): @@ -117,6 +232,8 @@ def validated(self) -> SandboxLaunch: _reject_disk_path(layer.path) require_file(self.scratch, "ext4 scratch image") _reject_disk_path(self.scratch) + if self.mount is not None: + self.mount.validated() return self def ordered_layers(self) -> tuple[SandboxLayer, ...]: @@ -140,6 +257,8 @@ def openvmm_arguments(self) -> list[str]: f"{self.workload_identity[0]}:{self.workload_identity[1]}", ) ) + if self.mount is not None: + arguments.extend(self.mount.openvmm_arguments()) return arguments def kernel_command_line(self, user_command_line: str = "") -> str: @@ -169,7 +288,14 @@ def kernel_command_line(self, user_command_line: str = "") -> str: if self.pids_max is not None: tokens.append(f"nvx_pids_max={self.pids_max + 1}") command_line = " ".join(token for token in tokens if token) - if len(command_line.encode("utf-8")) + 1 > SANDBOX_COMMAND_LINE_MAX_SIZE: + # OpenVMM appends the share's bootstrap tokens to the same x86 budget. + mount_tokens = ( + "" if self.mount is None else f" {self.mount.command_line_fragment()}" + ) + if ( + len(command_line.encode("utf-8")) + len(mount_tokens.encode("utf-8")) + 1 + > SANDBOX_COMMAND_LINE_MAX_SIZE + ): raise ScriptError( "sandbox kernel command line exceeds its 1024-byte x86 budget" ) diff --git a/scripts/nvx_tools/sandbox_lifecycle.py b/scripts/nvx_tools/sandbox_lifecycle.py index 44898581a..7d3d5b1f0 100644 --- a/scripts/nvx_tools/sandbox_lifecycle.py +++ b/scripts/nvx_tools/sandbox_lifecycle.py @@ -26,7 +26,7 @@ ControlSession, ManagedExecResult, ) -from .sandbox import SandboxLaunch, SandboxLayer +from .sandbox import SandboxLaunch, SandboxLayer, SandboxMount CONFIG_NAME = "config.json" RUNTIME_NAME = "runtime.json" @@ -201,9 +201,38 @@ def _serialize_launch( "network_proxy": network_proxy, "host_loopback_forward": list(host_loopback_forward), "cmdline": cmdline, + "mount": _serialize_mount(launch.mount), } +def _serialize_mount(mount: SandboxMount | None) -> dict[str, Any] | None: + if mount is None: + return None + resolved = mount.resolved() + return { + "guest_target": resolved.guest_target, + "host_path": os.fspath(resolved.host_path), + "access": resolved.access, + "denied_paths": [os.fspath(path) for path in resolved.denied_paths], + "owner": resolved.owner, + } + + +def _deserialize_mount(value: object) -> SandboxMount | None: + if value is None: + return None + if not isinstance(value, dict): + raise ScriptError("sandbox configuration is malformed") + mount = cast(dict[str, Any], value) + return SandboxMount( + guest_target=str(mount["guest_target"]), + host_path=Path(str(mount["host_path"])), + access=str(mount["access"]), + denied_paths=tuple(Path(str(path)) for path in mount["denied_paths"]), + owner=str(mount["owner"]), + ) + + def _deserialize_launch(config: dict[str, Any]) -> SandboxLaunch: try: layers = tuple( @@ -224,6 +253,8 @@ def _deserialize_launch(config: dict[str, Any]) -> SandboxLaunch: None if config["memory_max"] is None else int(config["memory_max"]) ), pids_max=None if config["pids_max"] is None else int(config["pids_max"]), + # State directories provisioned before live shares have no mount. + mount=_deserialize_mount(config.get("mount")), ) except (KeyError, TypeError, ValueError) as error: raise ScriptError("sandbox configuration is malformed") from error diff --git a/scripts/setup/README.md b/scripts/setup/README.md index 82e26b404..e84be7c1d 100644 --- a/scripts/setup/README.md +++ b/scripts/setup/README.md @@ -43,7 +43,9 @@ On Linux, omit `--runner-name` to prepare host dependencies without reconciling an existing runner's service or cache directory. Pass its name in a subsequent invocation to update the service; `--check-only` still validates runner state. Rustup bootstrap binaries are versioned and SHA-256 verified before execution; -Linux provisioning also installs `zstd` for native Actions cache archives. +Linux provisioning also installs `zstd` for native Actions cache archives and +`e2fsprogs` for the scratch image that the sandbox live-share microVM test +formats. Both runner setup scripts install a pinned, SHA-256-verified `sccache` binary. Runner services use a persistent `_work/_sccache` directory with a 10-GiB limit, disable Cargo incremental compilation, and expose `sccache` through diff --git a/scripts/setup/setup-linux-mshv.sh b/scripts/setup/setup-linux-mshv.sh index afcebff5f..5c662ec2e 100644 --- a/scripts/setup/setup-linux-mshv.sh +++ b/scripts/setup/setup-linux-mshv.sh @@ -59,14 +59,14 @@ install_packages() { run_as_root apt-get update run_as_root env DEBIAN_FRONTEND=noninteractive apt-get install -y \ bc binutils bison build-essential ca-certificates cmake cpio curl \ - docker-buildx docker.io flex git gzip iproute2 iptables \ + docker-buildx docker.io e2fsprogs flex git gzip iproute2 iptables \ libarchive-tools libelf-dev libssl-dev make ninja-build patch perl \ pkg-config protobuf-compiler python3 rsync tar util-linux xz-utils \ zstd elif command -v tdnf >/dev/null 2>&1; then run_as_root tdnf install -y \ bc binutils bison ca-certificates cmake cpio curl diffutils \ - docker-buildx docker-cli elfutils-libelf-devel findutils flex gcc \ + docker-buildx docker-cli e2fsprogs elfutils-libelf-devel findutils flex gcc \ gcc-c++ git glibc-devel glibc-iconv gzip iproute iptables \ kernel-headers libarchive libarchive-devel make moby-engine \ ninja-build openssl openssl-devel patch perl perl-FindBin \ @@ -76,7 +76,7 @@ install_packages() { run_as_root tdnf install -y docker-cli elif command -v dnf >/dev/null 2>&1; then run_as_root dnf install -y \ - bc binutils bison ca-certificates cmake cpio curl elfutils-libelf-devel \ + bc binutils bison ca-certificates cmake cpio curl e2fsprogs elfutils-libelf-devel \ findutils flex gcc gcc-c++ git glibc-devel gzip iproute iptables \ kernel-headers libarchive libarchive-devel make moby-engine \ ninja-build openssl openssl-devel patch perl pkgconf \ diff --git a/scripts/setup/setup-linux-runner.sh b/scripts/setup/setup-linux-runner.sh index e2c2c98b6..12773e3ef 100644 --- a/scripts/setup/setup-linux-runner.sh +++ b/scripts/setup/setup-linux-runner.sh @@ -402,14 +402,14 @@ install_packages() { run_as_root apt-get update run_as_root env DEBIAN_FRONTEND=noninteractive apt-get install -y \ bc binutils bison build-essential ca-certificates cmake cpio curl diffutils \ - flex git gzip iproute2 iptables \ + e2fsprogs flex git gzip iproute2 iptables \ libarchive-tools libelf-dev libssl-dev make ninja-build patch perl \ pkg-config protobuf-compiler python3 rsync tar util-linux xz-utils \ zstd elif command -v tdnf >/dev/null 2>&1; then run_as_root tdnf install -y \ bc binutils bison ca-certificates cmake cpio curl diffutils \ - elfutils-libelf-devel findutils flex gcc \ + e2fsprogs elfutils-libelf-devel findutils flex gcc \ gcc-c++ git glibc-devel glibc-iconv gzip icu iproute iptables \ kernel-headers libarchive libarchive-devel lttng-ust make \ ninja-build openssl openssl-devel patch perl perl-FindBin \ @@ -419,7 +419,7 @@ install_packages() { elif command -v dnf >/dev/null 2>&1; then run_as_root dnf install -y \ bc binutils bison ca-certificates cmake cpio curl diffutils \ - elfutils-libelf-devel findutils flex gcc gcc-c++ git glibc-devel \ + e2fsprogs elfutils-libelf-devel findutils flex gcc gcc-c++ git glibc-devel \ gzip iproute iptables kernel-headers libarchive libarchive-devel \ make ninja-build openssl openssl-devel patch perl \ pkgconf pkgconf-pkg-config protobuf-compiler python3 rsync \ diff --git a/scripts/test_microvm_tests.py b/scripts/test_microvm_tests.py index d311a3e57..5b11444a5 100644 --- a/scripts/test_microvm_tests.py +++ b/scripts/test_microvm_tests.py @@ -3,6 +3,7 @@ import argparse import json +import os import queue import shutil import socket @@ -1606,6 +1607,237 @@ def test_sandbox_blocks_use_fixed_roles_and_access(self): microvm_tests.SANDBOX_BLOCKS_COMPLETION_MARKER, ) + def test_sandbox_filesystem_scripts_check_the_share_before_success(self): + writable = microvm_tests._render_script( + "sandbox-filesystem.sh.in", + TARGET=microvm_tests.SANDBOX_FILESYSTEM_TARGET, + IDENTITY="1001:1001", + OWNER_CHECK='chmod 0600 "$share/guest-file" || fail 50', + ) + read_only = microvm_tests._render_script( + "sandbox-filesystem-read-only.sh.in", + TARGET=microvm_tests.SANDBOX_FILESYSTEM_TARGET, + ) + for script in (writable, read_only): + self.assertNotIn("@", script) + self.assertIn("share=/workspace\n", script) + self.assertIn('[ "$(id -u):$(id -g)" = 1001:1001 ]', writable) + self.assertLess( + writable.index("fail 50"), + writable.index(microvm_tests.SANDBOX_FILESYSTEM_WRITTEN_MARKER.decode()), + ) + self.assertLess( + writable.index("fail 52"), + writable.index(microvm_tests.SANDBOX_FILESYSTEM_OK_MARKER.decode()), + ) + self.assertLess( + read_only.index("fail 68"), + read_only.index(microvm_tests.SANDBOX_FILESYSTEM_READ_ONLY_MARKER.decode()), + ) + + def _run_sandbox_filesystem( + self, + root: Path, + outputs: dict[str, tuple[int, bytes]], + ) -> list[list[str]]: + layer = root / "ubuntu-distro.erofs" + layer.write_bytes(b"layer") + (root / "ubuntu-distro.erofs.manifest.json").write_text( + json.dumps({"uuid": "11111111-1111-1111-1111-111111111111"}), + encoding="utf-8", + ) + output_dir = root / "logs" + output_dir.mkdir() + identity: tuple[tuple[int, int], str] + if sys.platform == "win32": + identity = ((65534, 65534), "process") + else: + identity = ((os.getuid(), os.getgid()), "caller") + commands: list[list[str]] = [] + + def scratch(scratch_root: Path, _identity: tuple[int, int]) -> Path: + template = scratch_root / "scratch-template.ext4" + template.write_bytes(b"scratch") + return template + + def artifact(name: str) -> Path: + return root / name + + class FakeProcess: + def __init__(self, command: list[str], _log_path: Path) -> None: + commands.append(command) + mount = command[command.index("--mount") + 1].split(",") + self.target = mount[0] + self.export = Path(mount[1]) + self.access = mount[2] + probe = (self.export / "probe.sh").read_text(encoding="utf-8") + chmod = 'chmod 0600 "$share/guest-file" || fail 50\n' + caller_owned = cast(str, identity[1]) == "caller" + if (chmod in probe) != caller_owned: + raise AssertionError("probe chmod does not match the owner mode") + + def __enter__(self) -> "FakeProcess": + return self + + def __exit__(self, *_arguments: object) -> None: + return None + + def wait_for_line(self, marker: bytes, _timeout: float) -> None: + if marker == microvm_tests.SANDBOX_FILESYSTEM_WRITTEN_MARKER: + (self.export / "guest-dir").mkdir() + (self.export / "guest-dir" / "nested").write_bytes( + b"NVX-GUEST-NESTED\n" + ) + (self.export / "guest-file").write_bytes(b"NVX-GUEST-WRITE\n") + os.chmod(self.export / "guest-file", 0o600) + elif marker == microvm_tests.SANDBOX_FILESYSTEM_OK_MARKER: + if (self.export / "host-edit").read_bytes() != b"NVX-HOST-EDIT\n": + raise AssertionError("host edit was not staged") + + def wait(self, _timeout: float) -> openvmm_process.OpenvmmProcessResult: + command = commands[-1] + if "--microvm-report" in command: + Path(command[command.index("--microvm-report") + 1]).write_text( + json.dumps( + { + "schema_version": 1, + "instance_id": "0" * 32, + "backend": "whp", + "outcome": { + "operation": "run", + "category": "success", + "status_code": 0, + }, + "network_policy": { + "status": "not-configured", + "status_code": 0, + "mode": "none", + "allow_rule_count": 0, + "deny_rule_count": 0, + "host_loopback": "allow", + }, + "teardown": dict.fromkeys( + microvm_tests.OUTCOME_TEARDOWN_FIELDS, True + ), + } + ), + encoding="utf-8", + ) + returncode, output = outputs[f"{self.target},{self.access}"] + return openvmm_process.OpenvmmProcessResult(returncode, output) + + with ( + patch.object(microvm_tests, "artifact_path", side_effect=artifact), + patch.object( + microvm_tests, + "_sandbox_filesystem_identity", + return_value=identity, + ), + patch.object(microvm_tests, "_assign_sandbox_export_owner"), + patch.object( + microvm_tests, + "_sandbox_filesystem_scratch", + side_effect=scratch, + ), + patch.object(microvm_tests, "OpenvmmProcess", FakeProcess), + ): + microvm_tests.run_sandbox_filesystem( + Path("openvmm"), + Path("vmlinux"), + Path("initrd"), + "whp", + memory_mib=128, + timeout=60, + output_dir=output_dir, + ) + self.assertTrue((output_dir / "sandbox-filesystem.json").is_file()) + return commands + + def _sandbox_filesystem_outputs(self) -> dict[str, tuple[int, bytes]]: + return { + "/workspace,rw": ( + 0, + b"NVX-SANDBOX-VIRTFS: mounted microvm at /workspace (rw)\n" + b"NVX-SANDBOX-FILESYSTEM-WRITTEN\n" + b"NVX-SANDBOX-FILESYSTEM-OK\n" + b"NVX-SANDBOX-EXIT: status=0\n", + ), + "/workspace,ro": ( + 0, + b"NVX-SANDBOX-VIRTFS: mounted microvm at /workspace (ro)\n" + b"NVX-SANDBOX-FILESYSTEM-READ-ONLY-OK\n" + b"NVX-SANDBOX-EXIT: status=0\n", + ), + "/dev/nvx-share,ro": ( + 125, + b"NVX-SANDBOX-ERROR: virtio-fs target is reserved by the sandbox: " + b"/dev/nvx-share\n", + ), + "/bin/nvx-share,ro": ( + 125, + b"NVX-SANDBOX-ERROR: virtio-fs target traverses a symbolic link: " + b"/bin/nvx-share\n", + ), + } + + def test_sandbox_filesystem_checks_live_share_policy_and_rejections(self): + with tempfile.TemporaryDirectory() as temporary: + commands = self._run_sandbox_filesystem( + Path(temporary), self._sandbox_filesystem_outputs() + ) + + self.assertEqual( + [ + command[command.index("--mount") + 1].split(",")[0] + for command in commands + ], + ["/workspace", "/workspace", "/dev/nvx-share", "/bin/nvx-share"], + ) + expected_owner = "process" if sys.platform == "win32" else "caller" + for command, access in zip(commands[:2], ("rw", "ro"), strict=True): + self.assertTrue(command[command.index("--mount") + 1].endswith(access)) + self.assertTrue( + command[command.index("--mount-deny") + 1].endswith("secrets") + ) + self.assertEqual( + command[command.index("--mount-owner") + 1], expected_owner + ) + self.assertEqual(command[command.index("--memory") + 1], "256M") + self.assertEqual( + command[command.index("--microvm-lifecycle") + 1], "one-shot" + ) + cmdline = command[command.index("--cmdline") + 1] + self.assertIn("nvx_sandbox=1", cmdline) + self.assertIn("nvx_entrypoint=/bin/sh", cmdline) + self.assertIn( + "nvx_arg=/workspace/probe.sh", + commands[0][commands[0].index("--cmdline") + 1], + ) + self.assertIn("--microvm-report", commands[0]) + for command in commands[2:]: + self.assertNotIn("--mount-owner", command) + self.assertNotIn("--mount-deny", command) + + def test_sandbox_filesystem_rejects_an_accepted_unsafe_target(self): + outputs = self._sandbox_filesystem_outputs() + outputs["/bin/nvx-share,ro"] = (0, b"NVX-SANDBOX-READY: pid=2 layers=1\n") + with tempfile.TemporaryDirectory() as temporary: + with self.assertRaisesRegex(RuntimeError, "did not reject the symlink"): + self._run_sandbox_filesystem(Path(temporary), outputs) + + def test_sandbox_filesystem_requires_live_visibility_while_running(self): + outputs = self._sandbox_filesystem_outputs() + with ( + tempfile.TemporaryDirectory() as temporary, + patch.object( + microvm_tests, + "_check_sandbox_filesystem_writes", + side_effect=RuntimeError("not visible"), + ), + self.assertRaisesRegex(RuntimeError, "not visible"), + ): + self._run_sandbox_filesystem(Path(temporary), outputs) + def test_smp_snapshot_reruns_probe_and_checks_two_restores(self): with tempfile.TemporaryDirectory() as temporary: output_dir = Path(temporary) / "logs" @@ -2029,6 +2261,7 @@ def require(path: Path, _description: str) -> Path: for scenario in ( "console-snapshot", "sandbox-blocks", + "sandbox-filesystem", "scratch-snapshot", "snapshot-tiers", ): diff --git a/scripts/test_nvx_tools.py b/scripts/test_nvx_tools.py index aab6088bf..e285eafd2 100644 --- a/scripts/test_nvx_tools.py +++ b/scripts/test_nvx_tools.py @@ -642,8 +642,129 @@ def test_sandbox_command_parses_typed_launch_contract(self): self.assertEqual(args.memory_max, 268435456) self.assertEqual(args.pids_max, 64) self.assertEqual(args.workload_user, (1000, 1001)) + self.assertIsNone(args.mount) + self.assertEqual(args.mount_deny, []) + self.assertIsNone(args.mount_owner) self.assertIs(args.handler, nvx.command_sandbox) + def test_sandbox_forwards_a_live_mount_to_openvmm(self): + with tempfile.TemporaryDirectory() as temporary: + root = Path(temporary) + layer = root / "distro.erofs" + scratch = root / "scratch.ext4" + share = root / "share" + (share / "secrets").mkdir(parents=True) + layer.write_bytes(b"distro") + scratch.write_bytes(b"scratch") + base = [ + "sandbox", + "--layer", + f"distro,{layer},11111111-1111-1111-1111-111111111111", + "--scratch", + str(scratch), + "--workload-user", + "1001:1001", + "--mount", + f"/workspace,{share},rw", + "--mount-deny", + str(share / "secrets"), + "--dry-run", + ] + + def require(path: Path, _description: str) -> Path: + return path + + for extra, owner in ( + ([], sandbox.default_mount_owner()), + (["--mount-owner", "process"], "process"), + ): + with self.subTest(owner=owner): + with ( + patch.object(nvx, "require_file", side_effect=require), + patch.object( + nvx, "_format_command", return_value="formatted" + ) as format_command, + ): + nvx.command_sandbox(nvx.parse_args([*base, *extra])) + command = format_command.call_args.args[0] + self.assertEqual( + command[command.index("--mount") + 1], + f"/workspace,{share},rw", + ) + self.assertEqual( + command[command.index("--mount-deny") + 1], + str(share / "secrets"), + ) + self.assertEqual(command[command.index("--mount-owner") + 1], owner) + self.assertEqual( + command[command.index("--microvm-workload-identity") + 1], + "1001:1001", + ) + + def test_sandbox_mount_options_are_validated_before_launch(self): + layer_arguments = [ + "--layer", + "distro,distro.erofs,11111111-1111-1111-1111-111111111111", + "--scratch", + "scratch.ext4", + ] + for arguments, message in ( + ( + ["sandbox", *layer_arguments, "--mount-deny", "secrets"], + "require --mount", + ), + ( + ["sandbox", *layer_arguments, "--mount-owner", "process"], + "require --mount", + ), + ( + [ + "sandbox", + "exec", + "--state-dir", + "state", + "--mount", + "/workspace,share,rw", + ], + "only valid for sandbox run or provision", + ), + ( + ["sandbox", *layer_arguments, "--mount", "/proc/share,share,rw"], + "reserved by the sandbox", + ), + ): + with self.subTest(arguments=arguments): + with self.assertRaisesRegex(common.ScriptError, message): + nvx.command_sandbox(nvx.parse_args(arguments)) + + def test_run_forwards_mount_owner_only_with_a_mount(self): + args = nvx.parse_args( + [ + "run", + "--mount", + "/mnt/share,share,rw", + "--mount-owner", + "caller", + "--dry-run", + ] + ) + with ( + patch.object(nvx, "require_file", return_value=Path("artifact")), + patch.object( + nvx, "_format_command", return_value="formatted" + ) as format_command, + ): + nvx.command_run(args) + command = format_command.call_args.args[0] + self.assertEqual(command[command.index("--mount-owner") + 1], "caller") + + without_mount = nvx.parse_args(["run", "--mount-owner", "caller", "--dry-run"]) + with ( + patch.object(nvx, "require_file", return_value=Path("artifact")), + self.assertRaisesRegex(common.ScriptError, "requires --mount"), + ): + nvx.command_run(without_mount) + def test_sandbox_command_parses_state_aware_operations(self): provision = nvx.parse_args( [ @@ -4677,6 +4798,114 @@ def test_layer_parser_rejects_invalid_role_and_uuid(self): with self.assertRaisesRegex(common.ScriptError, "UUID is invalid"): sandbox.SandboxLayer.parse("distro,layer.erofs,not-a-uuid") + def test_mount_contract_forwards_live_share_and_counts_bootstrap_tokens(self): + distro = sandbox.SandboxLayer.parse( + "distro,distro.erofs,11111111-1111-1111-1111-111111111111" + ) + mount = sandbox.SandboxMount.parse( + "/workspace,share,rw", + denied_paths=(Path("share/secrets"),), + owner="process", + ) + self.assertEqual( + mount.openvmm_arguments(), + [ + "--mount", + f"/workspace,{Path('share')},rw", + "--mount-deny", + str(Path("share/secrets")), + "--mount-owner", + "process", + ], + ) + self.assertEqual(sandbox.SandboxMount.parse("/data,share").access, "ro") + self.assertEqual( + sandbox.SandboxMount.parse("/data,share").owner, + sandbox.default_mount_owner(), + ) + launch = sandbox.SandboxLaunch( + layers=(distro,), + scratch=Path("scratch.ext4"), + mount=mount, + ) + self.assertEqual( + launch.openvmm_arguments()[-6:], + mount.openvmm_arguments(), + ) + # The live-share bootstrap tokens are appended by OpenVMM, but they + # still consume the x86 command-line budget. + without_mount = sandbox.SandboxLaunch( + layers=(distro,), scratch=Path("scratch.ext4") + ) + padding = "x" * ( + sandbox.SANDBOX_COMMAND_LINE_MAX_SIZE + - len(without_mount.kernel_command_line()) + - 2 + ) + without_mount.kernel_command_line(padding) + with self.assertRaisesRegex(common.ScriptError, "1024-byte"): + launch.kernel_command_line(padding) + + def test_mount_contract_rejects_unsafe_targets_and_owners(self): + for value, message in ( + ("workspace,share,rw", "canonical absolute path"), + ("/,share,rw", "canonical absolute path"), + ("/work/../etc,share,rw", "canonical absolute path"), + ("/work//space,share,rw", "canonical absolute path"), + ("/work space,share,rw", "canonical absolute path"), + ("/work=space,share,rw", "canonical absolute path"), + ("/proc,share,rw", "reserved"), + ("/sys/share,share,rw", "reserved"), + ("/dev/shm,share,rw", "reserved"), + ("/.nvx-agent/share,share,rw", "reserved"), + ("/etc,share,rw", "reserved"), + ("/etc/machine-id,share,rw", "reserved"), + ("/workspace,share,write", "ro or rw"), + ("/workspace,,rw", "host path is empty"), + ("/workspace", "GUEST_TARGET,HOST_PATH"), + ): + with self.subTest(value=value): + with self.assertRaisesRegex(common.ScriptError, message): + sandbox.SandboxMount.parse(value, owner="process") + sandbox.SandboxMount.parse("/etc/app,share,rw", owner="process") + with self.assertRaisesRegex(common.ScriptError, "--mount-owner"): + sandbox.SandboxMount.parse("/workspace,share,rw", owner="root") + with self.assertRaisesRegex(common.ScriptError, "at most 128"): + sandbox.SandboxMount.parse( + "/workspace,share,rw", + denied_paths=tuple(Path(f"share/{index}") for index in range(129)), + owner="process", + ) + share = Path("share") + with ( + patch.object(sandbox.os, "name", "nt"), + self.assertRaisesRegex(common.ScriptError, "requires a Linux host"), + ): + sandbox.SandboxMount( + guest_target="/workspace", host_path=share, owner="caller" + ) + + def test_mount_contract_validates_host_paths_against_the_export(self): + with tempfile.TemporaryDirectory() as temporary: + share = Path(temporary) / "share" + (share / "secrets").mkdir(parents=True) + sandbox.SandboxMount( + guest_target="/workspace", + host_path=share, + denied_paths=(Path("secrets"), share / "secrets"), + ).validated() + with self.assertRaisesRegex(common.ScriptError, "does not exist"): + sandbox.SandboxMount( + guest_target="/workspace", + host_path=share, + denied_paths=(Path("missing"),), + ).validated() + with self.assertRaisesRegex(common.ScriptError, "directory does not"): + sandbox.SandboxMount( + guest_target="/workspace", + host_path=share / "missing", + ).validated() + def test_managed_lifecycle_provisions_and_deprovisions_state(self): with tempfile.TemporaryDirectory() as temporary: root = Path(temporary) @@ -4809,6 +5038,105 @@ def require(path: Path, _description: str) -> Path: ) session.ping.assert_called_once_with(10) + def test_managed_lifecycle_retains_the_live_mount(self): + with tempfile.TemporaryDirectory() as temporary: + root = Path(temporary).resolve() + layer_path = root / "distro.erofs" + scratch_path = root / "scratch.ext4" + share = root / "share" + (share / "secrets").mkdir(parents=True) + layer_path.write_bytes(b"layer") + scratch_path.write_bytes(b"scratch") + state = root / "state" + mount = sandbox.SandboxMount( + guest_target="/workspace", + host_path=share, + access="rw", + denied_paths=(Path("secrets"),), + owner="process", + ) + sandbox_lifecycle.provision( + state, + sandbox.SandboxLaunch( + layers=( + sandbox.SandboxLayer( + role="distro", + path=layer_path, + uuid="11111111-1111-1111-1111-111111111111", + ), + ), + scratch=scratch_path, + mount=mount, + ), + hypervisor="whp", + memory_mib=256, + net=None, + network_profile=None, + network_egress=None, + network_ingress=None, + network_egress_allow=(), + network_egress_deny=(), + host_loopback=None, + network_proxy=None, + host_loopback_forward=(), + cmdline="quiet", + ) + config = json.loads( + (state / sandbox_lifecycle.CONFIG_NAME).read_text(encoding="utf-8") + ) + self.assertEqual( + config["mount"], + { + "guest_target": "/workspace", + "host_path": str(share), + "access": "rw", + "denied_paths": ["secrets"], + "owner": "process", + }, + ) + self.assertEqual(sandbox_lifecycle._deserialize_launch(config).mount, mount) + legacy = dict(config) + del legacy["mount"] + self.assertIsNone(sandbox_lifecycle._deserialize_launch(legacy).mount) + with self.assertRaisesRegex(common.ScriptError, "malformed"): + sandbox_lifecycle._deserialize_launch({**config, "mount": []}) + + process = MagicMock() + process.pid = 123 + process.stdin = io.BytesIO() + context = MagicMock() + context.__enter__.return_value = MagicMock() + + def require(path: Path, _description: str) -> Path: + return path + + with ( + patch.object( + sandbox_lifecycle, + "require_file", + side_effect=require, + ), + patch.object( + sandbox_lifecycle.subprocess, + "Popen", + return_value=process, + ) as popen, + patch.object( + sandbox_lifecycle.ControlSession, + "connect", + return_value=context, + ), + ): + sandbox_lifecycle.start(state, 10) + + command = popen.call_args.args[0] + self.assertEqual( + command[command.index("--mount") + 1], + f"/workspace,{share},rw", + ) + self.assertEqual(command[command.index("--mount-deny") + 1], "secrets") + self.assertEqual(command[command.index("--mount-owner") + 1], "process") + def test_versioned_json_reader_supports_custom_version_field(self): with tempfile.TemporaryDirectory() as temporary: path = Path(temporary) / "outcome.json"