Mount the live virtio-fs share inside sandbox workloads

doc/design/current-limits.md

@@ -18,9 +18,23 @@ The current ABI family intentionally does not provide:
 - public activation of the reserved control console or an authenticated guest RPC broker;
 - replaceable sandbox configuration, a production OCI agent, or a public sandbox snapshot workflow;
 - serialization of live network flows or native host handles;
-- snapshotting of the contents of a live virtio-fs export; or
+- snapshotting of the contents of a live virtio-fs export;
+- more than one virtio-fs share, or different access modes within one share;
+- guest creation of symbolic links on the live virtio-fs share;
+- caller-owned virtio-fs requests on Windows, or propagation of the guest
+   caller's supplementary groups;
+- mounting a restore-time virtio-fs attachment inside a sandbox workload root; or
 - compatibility with standalone NVX `MVMSNAP*` or `WHPSNAP*` files.
 
+Sandbox mode does support the live virtio-fs share on a cold boot. The agent
+mounts it inside the workload root with `nosuid,nodev`, rejects targets that it
+manages or that traverse an image symbolic link, and fails closed if the mount
+fails. `--mount-deny` and read-only mode are enforced by the host server as in
+the standard profile. With `--mount-owner caller`, Linux hosts perform each
+request as the guest caller, map guest root to the export owner, and fail
+requests that cannot switch identity with `EPERM`. See
+[Live host share in the sandbox](../run.md#live-host-share-in-the-sandbox).
+
 The tier contract and post-restore gate are currently low-level OpenVMM and
 guest-agent primitives exercised by the native lifecycle suite. The public
 `nvx sandbox` command does not yet orchestrate platform builds, warm-shim

doc/design/sandbox-filesystem-and-agent-architecture.md

@@ -20,11 +20,11 @@ or snapshot-tier metadata alone.
 ## Implemented filesystem bootstrap
 
 The public `nvx sandbox` command accepts one to three role-bearing EROFS lower
-images, a preformatted ext4 scratch image, an absolute entrypoint, and
-individual argument tokens. It supplies non-secret kernel-command-line
-configuration; environment variables, secrets, arguments containing
-whitespace, and sandbox snapshot orchestration are not supported by this
-command. Lower-level OpenVMM capture and restore do support sandbox blocks.
+images, a preformatted ext4 scratch image, an absolute entrypoint, individual
+argument tokens, and an optional live virtio-fs share. It supplies non-secret
+kernel-command-line configuration; environment variables, secrets, arguments
+containing whitespace, and sandbox snapshot orchestration are not supported by
+this command. Lower-level OpenVMM capture and restore do support sandbox blocks.
 See [Run](../run.md#experimental-single-workload-sandbox) and
 [`scripts/nvx_tools/sandbox.py`](../../scripts/nvx_tools/sandbox.py).
 
@@ -46,10 +46,13 @@ device filter does not mean the current agent installs one.
 4. resolve and flush scratch, mount it as ext4 with `rw,nosuid,nodev`, and
    create its `upper` and `work` directories;
 5. mount overlayfs with top-first lower layers and `metacopy=on,xino=on`;
-6. start a child behind a FIFO barrier, place that child in the workload
+6. when OpenVMM attached a live virtio-fs share, validate its command-line
+   target, create any missing components without traversing image symbolic
+   links, and mount the share inside the overlay with `nosuid,nodev`;
+7. start a child behind a FIFO barrier, place that child in the workload
    cgroup before releasing it, and retain a runtime-tmpfs machine-ID file;
-7. wait for the child, unmount the overlay, layers, and scratch, and return its
-   status through the guest exit helper.
+8. wait for the child, unmount the share, overlay, layers, and scratch, and
+   return its status through the guest exit helper.
 
 The assembled view is:
 

doc/design/validation.md

@@ -36,6 +36,13 @@ pre-entry rejection of missing, corrupt, mismatched, or wrong-geometry media.
 Denied-filesystem coverage verifies listing suppression, allowed writes,
 direct and parent-relative denial, symlink/junction alias denial, a second
 virtio-fs mount, and pre-boot rejection of unsafe path policies.
+Sandbox live-share coverage boots the real sandbox agent over the Ubuntu EROFS
+layer with a read-write share. While the workload runs, it verifies host-file
+visibility, denied-path hiding, nested workload writes and `chmod` that appear
+on the host immediately, and a host edit that the workload observes. On Linux
+it also verifies that caller-owned requests create those files as the export
+owner. It then verifies a read-only share that rejects writes, and agent
+rejection of reserved and symlinked mount targets before the workload starts.
 The native suite targets KVM, MSHV, and WHP; a passing run on one backend is
 not a fresh result for the others.
 Coverage also includes

doc/run.md

@@ -252,7 +252,9 @@ backing and are onlined before restore readiness.
 ## virtio-fs host mapping
 
 The microVM reserves one mapping slot with a fixed `microvm` tag. On a cold
-boot with `--mount`, the initramfs mounts it automatically:
+boot with `--mount`, the initramfs mounts it automatically. In the sandbox
+profile, the agent instead mounts it inside the workload root; see
+[Live host share in the sandbox](#live-host-share-in-the-sandbox):
 
 ```bash
 python3 scripts/nvx.py run --mount "/mnt/host,/absolute/host/share,rw"
@@ -283,6 +285,44 @@ mkdir -p /mnt/host
 mount -t virtiofs microvm /mnt/host
 ```
 
+The share is live host state with no guest caching: entry and attribute cache
+lifetimes are zero and file I/O is direct. Host and guest changes are visible to
+the other side immediately. In exchange, every path lookup is a round trip to
+the host, which metadata-heavy workloads such as package installs will notice.
+Files appear exactly as they are on the host, and NVX strips nothing from a live
+share. Hide credentials and other sensitive paths with `--mount-deny`. The
+device adds no network path, so the network and egress policy are unchanged.
+
+By default, OpenVMM performs every guest request as its own host identity, so
+guest-created files are owned by the OpenVMM user. On Linux,
+`--mount-owner caller` performs each request as the guest caller's UID and GID
+instead, and maps guest root to the owner of the export root:
+
+```bash
+python3 scripts/nvx.py run --mount "/mnt/host,/absolute/host/share,rw" \
+  --mount-owner caller
+```
+
+Caller mode rejects an export root owned by UID 0 or GID 0. Each request runs
+without OpenVMM's supplementary groups and effective capabilities, so a
+privileged OpenVMM process cannot lend the guest its privileges. OpenVMM must
+run as the export owner or hold `CAP_SETUID` and `CAP_SETGID`. Without those
+capabilities, only requests from OpenVMM's own identity, including guest root,
+succeed, and they keep OpenVMM's supplementary groups; any other caller gets
+`EPERM`. Supplementary groups of the guest caller are not propagated. Because
+the guest names the caller, guest root can act as any non-root host user inside
+the export. Grant the capabilities only for exports that hold no other users'
+files. Windows rejects `--mount-owner caller`: files are created by the OpenVMM
+user, and the guest sees attributes derived from that user's access. The owner
+mode is host policy rather than snapshot state, so a restore selects it again.
+
+The microVM has exactly one share, with a single access mode. For example, a
+read-write workspace and a read-only tool cache cannot be exported as two
+shares. A second share would need a new microVM ABI version. Guest creation of
+symbolic links returns `ENOTSUP`, so tools that create links, such as
+`npm ci` with `node_modules/.bin`, must run in scratch rather than in the
+share. Existing host symbolic links are visible and are resolved by the guest.
+
 ## Experimental single-workload sandbox
 
 The `sandbox` command launches the microVM with one to three compressed
@@ -336,6 +376,58 @@ The outer agent retains the initramfs root; the capability-stripped child
 enters only the assembled root with `chroot`, because Linux cannot
 `pivot_root` away from an initramfs `rootfs`.
 
+### Live host share in the sandbox
+
+Add `--mount GUEST_TARGET,HOST_PATH[,ro|rw]` to `sandbox run` or
+`sandbox provision` to live-share one host directory with the workload:
+
+```bash
+python3 scripts/nvx.py sandbox \
+  --layer distro,build/ubuntu-distro.erofs,11111111-1111-1111-1111-111111111111 \
+  --scratch /var/lib/nvx/scratch.ext4 \
+  --workload-user 1001:1001 \
+  --mount /workspace,/home/runner/work/repo,rw \
+  --mount-deny /home/runner/work/repo/.secrets
+```
+
+After it assembles the overlay, and before it starts any workload, the agent
+mounts the share at `GUEST_TARGET` inside the workload root with the selected
+mode and `nosuid,nodev`. The mount lives below the overlay, so it survives the
+workload's private mount namespace and `chroot`. The workload itself still has
+no capabilities, so it cannot mount or unmount the share. The share exists only
+in the workload root; the agent does not mount it anywhere else. A one-shot
+agent unmounts it before the overlay. The agent fails closed with
+`NVX-SANDBOX-ERROR` and status 125, never starting the workload, if any of
+these holds:
+
+- the target is not canonical;
+- the target is `/etc` or `/etc/machine-id`, or lies under `/proc`, `/sys`,
+  `/dev`, or `/.nvx-agent`, all of which the agent manages;
+- a component of the target in the image is a symbolic link or a
+  non-directory;
+- the mount fails.
+
+Missing target components are created as root-owned `0755` directories in
+scratch. The marker `NVX-SANDBOX-VIRTFS: mounted microvm at GUEST_TARGET (MODE)`
+reports a successful mount. Without `--mount`, the sandbox is unchanged.
+
+The workload accesses the share as its fixed `--workload-user` identity. The
+guest kernel checks access against the host owners and modes, so choose the
+identity that owns the exported directory, such as the runner user; the image
+must define that user, as for any `--workload-user`. On Linux, the command
+passes `--mount-owner caller` by default. OpenVMM then performs workload
+requests as that identity and agent requests as the export owner, so with an
+identity that owns the export, every file the workload creates on the share is
+owned by that host user. The export root must not be owned by UID 0 or GID 0.
+OpenVMM must run as the export owner or hold `CAP_SETUID` and `CAP_SETGID`;
+otherwise the workload's share requests fail with `EPERM`. Use
+`--mount-owner process` to perform every request as the OpenVMM process
+instead. Windows supports only `process`. A
+read-only share rejects workload writes with `EROFS`. `--mount-deny` paths stay
+hidden inside the container. The policy, caching, and link limits in
+[virtio-fs host mapping](#virtio-fs-host-mapping) apply unchanged. A sandbox
+snapshot restore does not mount a newly attached share for the workload.
+
 For a state-aware sandbox, provision configuration without starting a VM,
 start it once, run multiple workloads in the same warm guest, stop it while
 retaining configuration, and finally deprovision it:

doc/usage.md

@@ -200,11 +200,15 @@ python3 scripts/nvx.py test-microvm
 Runs NVX-owned Linux, SMP, virtio, sandbox, and snapshot correctness scenarios
 against the public OpenVMM CLI. Repeat `--scenario` to select a subset; without
 it, every scenario supported by the selected guest runs. Alpine remains the
-default. Ubuntu rejects the Alpine-control-only `sandbox-blocks` and
-`scratch-snapshot` scenarios, the Alpine-prompt-specific `console-snapshot`
-scenario, and the sandbox-control-dependent `snapshot-tiers` scenario. The
-command requires `build/vmlinux`, the selected initramfs, and
-`openvmm/target/release/openvmm[.exe]`.
+default. Ubuntu rejects the Alpine-control-only `sandbox-blocks`,
+`sandbox-filesystem`, and `scratch-snapshot` scenarios, the
+Alpine-prompt-specific `console-snapshot` scenario, and the
+sandbox-control-dependent `snapshot-tiers` scenario. The command requires
+`build/vmlinux`, the selected initramfs, and
+`openvmm/target/release/openvmm[.exe]`. The `sandbox-filesystem` scenario also
+requires `build/ubuntu-distro.erofs` and its manifest. On Linux it formats its
+scratch image with `mkfs.ext4`; on Windows it copies
+`build/ubuntu-smoke-scratch.ext4`.
 
 ### `test-adversarial`
 
@@ -296,6 +300,8 @@ python3 scripts/nvx.py run
     [--memory-capacity-mib MIB]
     [--processors {1,2,4,8}]
     [--mount GUEST_TARGET,HOST_PATH[,ro|rw]]
+    [--mount-deny HOST_PATH]...
+    [--mount-owner {process,caller}]
     [--net IPV4/PREFIX]
     [--network-profile {portable}]
     [--network-egress {allow,deny}]
@@ -322,6 +328,8 @@ python3 scripts/nvx.py run
 | `--memory-capacity-mib MIB` | none | Reserve an immutable, 128 MiB-aligned RAM capacity for a fresh microVM snapshot. |
 | `--processors {1,2,4,8}` | `1` | Select the microVM processor count. |
 | `--mount GUEST_TARGET,HOST_PATH[,ro\|rw]` | none | Expose one host directory to the absolute guest target. Active snapshot restore requires the same canonical path, target, and mode; a dormant-slot restore may attach a new mapping that the resumed guest mounts explicitly. |
+| `--mount-deny HOST_PATH` | none | Hide an existing path inside the `--mount` export. Repeat for multiple paths; relative paths are resolved against the export root. |
+| `--mount-owner {process,caller}` | OpenVMM default (`process`) | Select the host identity for guest requests on `--mount`. `caller` is Linux-only; see [virtio-fs host mapping](run.md#virtio-fs-host-mapping). |
 | `--net IPV4/PREFIX` | none | Enable virtio-net with the static guest IPv4 address and prefix. |
 | `--network-profile {portable}` | none | Select the required cross-platform network behavior contract; must be specified with `--net`. |
 | `--network-egress {allow,deny}` | `allow` | Set the default guest egress policy. |
@@ -354,6 +362,9 @@ python3 scripts/nvx.py sandbox
     [--hostname NAME]
     [--memory-max BYTES]
     [--pids-max COUNT]
+    [--mount GUEST_TARGET,HOST_PATH[,ro|rw]]
+    [--mount-deny HOST_PATH]...
+    [--mount-owner {process,caller}]
     [--memory-mib MIB]
     [--hypervisor {auto,whp,kvm,mshv}]
     [--net IPV4/PREFIX]
@@ -371,6 +382,9 @@ python3 scripts/nvx.py sandbox
 | `--hostname NAME` | `nvx-sandbox` | Set the workload UTS hostname. |
 | `--memory-max BYTES` | none | Set the workload cgroup memory limit. |
 | `--pids-max COUNT` | none | Set the workload cgroup process limit. |
+| `--mount GUEST_TARGET,HOST_PATH[,ro\|rw]` | none | Live-share one host directory at the absolute target inside the workload root; valid for `run` and `provision`. The target cannot be `/etc`, `/etc/machine-id`, or a path under `/proc`, `/sys`, `/dev`, or `/.nvx-agent`. |
+| `--mount-deny HOST_PATH` | none | Hide an existing path inside the export. Repeat for multiple paths; relative paths are resolved against the export root. |
+| `--mount-owner {process,caller}` | `caller` on Linux, `process` on Windows | Select the host identity for workload requests on the share. |
 | `--memory-mib MIB` | `256` | Set guest memory in MiB. |
 | `--hypervisor {auto,whp,kvm,mshv}` | `auto` | Select the host hypervisor. |
 | `--net IPV4/PREFIX` | none | Enable virtio-net with a static guest address. |

guest/common/init

@@ -111,6 +111,7 @@ fi
 mount_virtfs() {
     /sbin/nvx-hostmount || fatal "virtfs: failed to mount live host directory"
 }
+# The sandbox agent mounts the share inside the container root instead.
 if [ "$sandbox_mode" = false ]; then
     mount_virtfs
 fi

guest/common/nvx-init-agent

@@ -126,6 +126,60 @@ mount_layer() {
     fi
 }
 
+# Mounts the optional live virtio-fs share inside the container root. The
+# workload cannot mount it itself because it runs without capabilities, and a
+# mount below the overlay survives the private mount namespace and chroot.
+mount_virtfs_share() {
+    virtfs_tag=$(cmdline_value virtfs_tag || true)
+    [ -n "$virtfs_tag" ] || return 0
+    [ "$virtfs_tag" = microvm ] || fatal "unexpected virtio-fs tag: $virtfs_tag"
+    virtfs_dir=$(cmdline_value virtfs_dir || true)
+    virtfs_mode=$(cmdline_value virtfs_mode || true)
+    case "$virtfs_mode" in
+        ro | rw) ;;
+        *) fatal "invalid virtio-fs mode: $virtfs_mode" ;;
+    esac
+    case "$virtfs_dir" in
+        /*) ;;
+        *) fatal "virtio-fs target must be an absolute path: $virtfs_dir" ;;
+    esac
+    # The appended slash also catches the root and trailing dot components.
+    case "$virtfs_dir/" in
+        *//* | */./* | */../*)
+            fatal "virtio-fs target is not a canonical path: $virtfs_dir"
+            ;;
+        /proc/* | /sys/* | /dev/* | /.nvx-agent/* | /etc/ | /etc/machine-id/)
+            fatal "virtio-fs target is reserved by the sandbox: $virtfs_dir"
+            ;;
+    esac
+
+    # Nothing else runs yet, so checking and creating each component in turn
+    # cannot race with an image symlink that redirects the mount outside the
+    # container root.
+    virtfs_path=$rootfs
+    virtfs_remainder=${virtfs_dir#/}
+    while [ -n "$virtfs_remainder" ]; do
+        virtfs_component=${virtfs_remainder%%/*}
+        case "$virtfs_remainder" in
+            */*) virtfs_remainder=${virtfs_remainder#*/} ;;
+            *) virtfs_remainder= ;;
+        esac
+        virtfs_path=$virtfs_path/$virtfs_component
+        [ ! -L "$virtfs_path" ] ||
+            fatal "virtio-fs target traverses a symbolic link: $virtfs_dir"
+        if [ ! -e "$virtfs_path" ]; then
+            mkdir -m 0755 "$virtfs_path" ||
+                fatal "failed to create virtio-fs target: $virtfs_dir"
+        fi
+        [ -d "$virtfs_path" ] && [ ! -L "$virtfs_path" ] ||
+            fatal "virtio-fs target is not a directory: $virtfs_dir"
+    done
+    mount -t virtiofs -o "$virtfs_mode,nosuid,nodev" "$virtfs_tag" "$virtfs_path" ||
+        fatal "failed to mount the virtio-fs share at $virtfs_dir"
+    virtfs_target=$virtfs_path
+    echo "NVX-SANDBOX-VIRTFS: mounted $virtfs_tag at $virtfs_dir ($virtfs_mode)"
+}
+
 mkdir -p "$layers" "$scratch" "$rootfs"
 mount -t tmpfs -o mode=755,nosuid,nodev tmpfs /run ||
     fatal "failed to mount runtime tmpfs"
@@ -208,6 +262,8 @@ mkdir -p "$scratch/upper" "$scratch/work"
 mount -t overlay overlay \
     -o "lowerdir=$lowerdirs,upperdir=$scratch/upper,workdir=$scratch/work,metacopy=on,xino=on" \
     "$rootfs" || fatal "failed to assemble the container overlay"
+virtfs_target=
+mount_virtfs_share
 
 entrypoint=$(cmdline_value nvx_entrypoint || true)
 [ -n "$entrypoint" ] || entrypoint=/bin/sh
@@ -291,6 +347,10 @@ echo "NVX-SANDBOX-READY: pid=$container_pid layers=$layer_count"
 status=0
 wait "$container_pid" || status=$?
 rm -f "$runtime/container.pid" "$runtime/workload-machine-id"
+if [ -n "$virtfs_target" ] && ! umount "$virtfs_target"; then
+    echo "NVX-SANDBOX-ERROR: failed to unmount the virtio-fs share" >&2
+    status=1
+fi
 if ! umount "$rootfs"; then
     echo "NVX-SANDBOX-ERROR: failed to unmount the container overlay" >&2
     status=1