fastcontainer - Minimal btrfs + systemd-nspawn layered container builder

Installation

# 1. Clone the repository
git clone https://github.com/paulalesius/fastcontainer.git
cd fastcontainer

# 2. Recommended: install with uv
uv sync --force-reinstall

Design Philosophy — Built for R&D, not production hardening

fastcontainer is intentionally minimal. It gives you fast, reproducible, layered containers using only btrfs snapshots and systemd-nspawn. No Dockerfiles, no OCI images, no daemon, no registry — just plain directories on a btrfs filesystem that you can inspect, modify, or delete with normal tools.

Perfect for machine-learning research, GPU-heavy experiments, and any workflow where you want full control and instant rebuilds of intermediate layers.

Quick Start

sudo fastcontainer build <containers_dir> <prepare.yaml> -p <profile> [-v] [--prune] [-s] [-b] [--dry-run] [-D KEY=VALUE]... [-- <command...>]

Per-Step User (RUN(user): / USE(user): / cmd(user):)

You can now specify which user a build step or post-build command runs as:

steps:
  - RUN(root): |
      whoami > /as-root.txt

  - RUN(noname): |
      whoami > /as-noname.txt
      id

  - RUN({{MYUSER}}): |      # variables are fully supported
      echo "running as {{MYUSER}}"

  - USE(root): my-snippet   # snippet runs as root

  - RUN: |                  # plain RUN: still defaults to root
      echo "plain RUN is root"

For the final command:

cmd(noname): |
  echo "=== Starting benchmark as $(whoami) ==="
  /llama.cpp/build/bin/llama-bench ...

# or keep the old style (still works)
cmd: |
  echo "this runs as root"

Important rules:

• The default user is always root.
• You cannot use --user or -u in any add: section anymore (fastcontainer will raise a clear error). The user is now controlled only per-step.
• The debug shell on build failure now automatically runs as the same user as the failing step.
• The interactive shell (-s) on success runs as the cmd(user) you defined.
• User changes are part of the cache fingerprint — changing the user forces a rebuild of that layer.

Interactive Shell (-s / --shell) — New in v0.7.0

Pass -s (or --shell) to drop into an interactive shell:

• On build failure → shell opens inside the temporary failing layer (you can debug/fix things).
• On successful build → shell opens inside an ephemeral copy of the final image instead of running cmd: (nothing you do in it is persisted).

The shell always respects the user defined for that step/profile.

Variables (-D) and env: — New in v0.9.1

All variables used with {{VAR}} must now be explicitly declared in a top-level env: section (in the current YAML or any import-base:).

env:
  HOST_CACHE: /tmp/default-cache          # default value
  NVIDIA_DRIVER_VERSION: 595.58.03
  USER_HOME_DIR: /home/noname
  CACHE_STORE_DIR: /data/fastcontainer-cache

base:
  create: |
    debootstrap --cache-dir={{CACHE_STORE_DIR}}/debootstrap ...

profiles:
  common:
    add:
      - "--bind={{CACHE_STORE_DIR}}/apt-cache:/var/cache/apt"
      - "--bind={{HOST_CACHE}}:/root/.cache"

Rules:

• {{VAR}} can only be used for variables listed in env: (anywhere in the inheritance/import tree).
• -D KEY=VALUE on the command line can only override variables that are declared in env:.
• Local env: overrides imported env:.
• Clear error messages are shown for:
  • Using an undeclared {{VAR}}
  • Passing a -D for a variable that is not in any env:

Example command:

sudo fastcontainer build ... -p default \
  -D HOST_CACHE=/home/noname/.cache \
  -D CACHE_STORE_DIR=/my/custom/cache

{{VAR}} substitution is now supported inside env: values (with chaining, any definition order, and -D overrides).
Variables are also supported in base.create:, base.add:, add:, steps:, cmd:, check:, and snippets.

Example of env: variable chaining:

env:
  BASE_DIR: /data/fastcontainer
  CACHE_DIR: "{{BASE_DIR}}/cache"
  LOG_DIR: "{{CACHE_DIR}}/logs"

### Profiles & Inheritance

```yaml
profiles:
  common:
    add:
      - "--tmpfs=/var/tmp"
      - "--private-users=no"
      - "--resolv-conf=replace-stub"
      - "--timezone=off"
    steps:
      - RUN: |
          apt-get update && apt-get install -y ...

  cuda:
    extend: common          # inherits add: + steps: from common
    steps:
      - RUN: |
          # CUDA-specific steps here

Base Import (import-base:) — New in v0.9.0+

Extract common bases, snippets, and profiles into reusable library files.
Supports full chaining — you can have three.yaml → two.yaml → one.yaml (and so on).

Library file (one.yaml):

base:
  name: ubuntu-noble-minimal
  create: |
    debootstrap --variant=minbase noble . http://archive.ubuntu.com/ubuntu/

profiles:
  common:
    add:
      - "--tmpfs=/var/tmp"
      - "--private-users=no"
      - "--resolv-conf=replace-stub"
      - "--timezone=off"
    steps:
      - RUN: |
          apt-get update
          apt install -y software-properties-common wget git ...

Intermediate file (two.yaml):

import-base: one.yaml
profiles:
  cuda:
    extend: common
    steps:
      - RUN: |
          # CUDA-specific steps...

Project file (three.yaml):

import-base: two.yaml

profiles:
  run-llama-cpp:
    extend: cuda
    # your final add: / cmd: / steps: ...

Rules (updated):

• Use either base: or import-base:, never both (clear error if both are present).
• The deepest file in the chain must contain a base: section.
• profiles:, snippets:, and env: are merged recursively. Local values always override imported ones.
• extend: works transparently across the entire import chain.
• Paths are resolved relative to the importing YAML file.
• Circular imports are detected and rejected with a clear error.
• Arbitrary depth is supported (not limited to one level).

This makes large projects dramatically shorter while keeping all the shared knowledge in one maintainable place.

Reusable Snippets (snippets: + USE:)

snippets:
  build-llama:
    RUN: |
      git clone https://github.com/ggml-org/llama.cpp.git
      cd llama.cpp
      cmake -B build -DGGML_CUDA=ON && cmake --build build --config Release -j $(nproc)

profiles:
  llama-cpp:
    steps:
      - USE(root): build-llama     # runs the snippet as root

Post-build command (cmd:)

Define a default command that runs automatically after a successful build:

cmd(noname): |
  echo "=== Starting llama-bench as $(whoami) ==="
  /llama.cpp/build/bin/llama-bench ...

You can also pass a trailing command on the CLI:

sudo fastcontainer build ... -p myprofile -- echo "one-off command"

Important: The cmd: (and any trailing command) runs in an ephemeral container (--ephemeral). Any changes made are discarded after it finishes. The final cached image is never modified.

Examples

# Basic build
sudo fastcontainer build /disk/fastcontainer ./sample/sample.yaml -p default \
  -D HOST_CACHE=/home/noname/.cache

# Full GPU + runtime variant (using import-base)
sudo fastcontainer build /disk/fastcontainer ./sample/ubuntu24.04-cu132-llama-cpp.yaml -p run-llama-cpp

# Build and immediately drop into a shell (success or failure)
sudo fastcontainer build ... -p default -s

# Verbose build
sudo fastcontainer build ... -v

Final image name format:
<effective_base>-<profile>-<40hex_fingerprint>

Caching, the check: gate and --prune

Builds are incremental and every artifact is content-addressed:

• Layer cache. Each step's layer is hashed from the previous layer's hash, the step command, the step user and the effective nspawn flags. An unchanged step reuses its cached layer; changing a command, a user, or any earlier step rebuilds that step and everything after it.
• Final fingerprint. The <40hex_fingerprint> covers every step (command + user), the nspawn flags, check: and cmd: / cmd(user):.
• check: gate. When the final image already exists and the profile defines check:, the check runs in an ephemeral copy of the cached image (it can never modify it). Pass → the image is reused as-is and no steps re-run. Fail → the cached image is deleted and the profile is deep-rebuilt: every step of that profile is re-executed from scratch, ignoring the layer cache, and so is every profile that extends it (its cached layers sit on top of the changed ones). Without check:, an existing image is always re-created from its (cached) layers.
• --prune deletes only the intermediate layers used by this build; layers belonging to other profiles of the same base are kept.

Concurrent builds

Multiple fastcontainer build invocations may run at the same time in the same containers_dir — including two builds of the same profile (the second one waits, then reuses the first one's cached layers). Coordination is per-resource, not global:

• Per-resource locks. Each build computes its full layer plan up front (layer names are content-addressed, so every path it will ever touch is known before anything executes), then takes one flock per store path — the base, each layer, each final image — in sorted order. A single global acquisition order makes deadlocks impossible; unrelated builds (different bases, different profiles) never block each other.
• Atomic image replacement. Final images are built in a temp subvolume and moved into place with rename(2), so anything running on the image (systemd-nspawn -D <image>) sees either the old or the new one, never a half-built one.
• Crash-safe cleanup. Every build holds a liveness flock for its lifetime and publishes the temp subvolumes it owns. A temp is swept as stale only if no live build claims it — a crashed build's flock is released by the kernel, so this needs no timeouts and can never delete another running build's work.

Lock and liveness files live in .fastcontainer-locks/ inside the store. They are intentionally never deleted (removing a lock file while another process is waiting on it is what breaks locking) — a store that has seen many different layers accumulates lock files over time; they are tiny and harmless. Locking only works between processes on the same local filesystem (flocks are local by nature), which is exactly the supported setup: one btrfs volume on one machine.

Other features

• Per-step users (RUN(user):, USE(user):, cmd(user):) — v0.8.0
• Base library import via import-base: — v0.9.0
• -s / --shell: interactive shell on failure or success — v0.7.0
• --prune: delete the intermediate layers used by this build (other profiles of the same base keep theirs)
• Concurrent builds — multiple builds may run at the same time in the same store (see below)
• Every layer contains a fastcontainer.json manifest with full build history
• Reusable snippets: + USE: syntax
• -b / --boot: run the post-build cmd: (or the -s shell) with systemd-nspawn --boot (full machine, init/PID 1)
• --dry-run: run the full pipeline (config parsing, env, inheritance, layer plan) without root, btrfs or nspawn

Contributing & Development

# Regenerate the full source prompt for AI assistance
bash scripts/project-to-prompt.sh

---

Happy container building!
If you have questions or ideas for the next feature, open an issue or just ping me.