Add a real sub-interpreter backend with a pre-warmed cell pool

.github/workflows/ci.yml

@@ -34,7 +34,7 @@ jobs:
       fail-fast: false
       matrix:
         os: [ubuntu-22.04, ubuntu-24.04]
-        python-version: ["3.11", "3.12", "3.13"]
+        python-version: ["3.11", "3.12", "3.13", "3.14"]
     steps:
       - uses: actions/checkout@v4
       - uses: actions/setup-python@v5
@@ -101,6 +101,40 @@ jobs:
         continue-on-error: true
         run: pytest -q tests/test_matrix_hardening.py -m "race and no_gil"
 
+  tests-subinterpreter:
+    name: sub-interpreter cells / py3.14t
+    runs-on: ubuntu-24.04
+    steps:
+      - uses: actions/checkout@v4
+      - uses: actions/setup-python@v5
+        with:
+          python-version: "3.14t"
+      - name: Install dependencies
+        # Same shape as the 3.13t job: install without the runtime deps that
+        # may not have free-threaded wheels yet, then re-add the pure-Python
+        # ones the import path needs. The suite's crypto stub (tests/conftest.py)
+        # covers the rest, and nothing in this job's scope needs real crypto.
+        run: |
+          python -m pip install -e . --no-deps
+          python -m pip install pytest pyyaml platformdirs
+      - name: Verify this build actually has sub-interpreters
+        # Every sub-interpreter test skips itself when the build cannot run it.
+        # That is right for the 3.11-3.13 matrix, but it means this job would
+        # report green on a runner that silently resolved to an older Python
+        # while testing nothing. Fail loudly instead.
+        run: |
+          python - <<'PY'
+          import sys
+          from pyisolate.runtime import subinterpreter as s
+          assert s.is_available(), f"no concurrent.interpreters on {sys.version}"
+          assert not sys._is_gil_enabled(), "expected a free-threaded build"
+          print("ok:", sys.version)
+          PY
+      - name: Run the sub-interpreter backend suite
+        run: pytest -q tests/test_subinterpreter_backend.py tests/test_supervisor.py
+      - name: Validate the no-GIL readiness axis on 3.14t
+        run: pytest -q tests/test_nogil.py
+
   tests-soak:
     name: soak / 2k spawn-kill cycles
     if: github.event_name == 'schedule'

CHANGELOG.md

@@ -8,6 +8,13 @@ guarantees; **no release should be treated as a hardened security boundary**.
 ## [Unreleased]
 
 ### Added
+- `backend="subinterpreter"`: real CPython sub-interpreter cells on 3.14+, via
+  `concurrent.interpreters`, with a pre-warmed `CellPool`. Each guest gets its
+  own `sys.modules` and its own `builtins`, so the import allow-list is a
+  property of the interpreter rather than thread-local state. Still an
+  execution cell, not a boundary against hostile Python. Fails closed below
+  3.14 rather than degrading to the thread backend.
+
 - `backend="process"` boundary mode: a real separate-process boundary confined
   by `no_new_privs` + a seccomp deny-list, Landlock filesystem rules, Landlock
   TCP-egress rules (Landlock ABI ≥ 4), a coarse per-cgroup eBPF/LSM deny-mask,
@@ -24,18 +31,28 @@ guarantees; **no release should be treated as a hardened security boundary**.
 - `pyisolate[operator]` optional-dependency group for the Kubernetes operator.
 
 ### Changed
+- CI covers CPython 3.14: the unit matrix gains `3.14`, and a new
+  `sub-interpreter cells / py3.14t` job runs the sub-interpreter backend on a
+  free-threaded build. That job asserts the interpreter really is a
+  free-threaded 3.14 before running anything, because every sub-interpreter
+  test skips itself when the build cannot run it -- correct for the 3.11-3.13
+  matrix, but it would otherwise let the job report green having tested
+  nothing.
 - `backend="subinterpreter"` is renamed to `backend="thread"`, which is what it
-  has always run. The old spelling still resolves and emits a
-  `DeprecationWarning`; it is reserved for a real CPython sub-interpreter
-  backend rather than kept as a permanent synonym, so callers who want the
-  thread runtime should pass `"thread"`. `DEPRECATED_BACKEND_ALIASES` is
-  exported alongside `SUPPORTED_BACKENDS`.
+  has always run, and the `subinterpreter` name now selects the real
+  sub-interpreter backend. `DEPRECATED_BACKEND_ALIASES` is exported alongside
+  `SUPPORTED_BACKENDS` and is currently empty.
 - Threat model and `SECURITY.md` reconciled with the real, backend-conditional
   boundary (the sub-interpreter backend is an execution cell, not a boundary
   against hostile Python).
 
 ### Known gaps
 - The broker `request` op is surfaced but not yet executed end-to-end.
+- A running sub-interpreter cell cannot be reclaimed: one that overruns its
+  wall-time deadline is abandoned, and its thread stays pinned until the
+  process exits. Cells enforce a wall-time deadline and no other quota;
+  `sys.getallocatedblocks()` is process-global, so per-cell memory
+  accounting needs a worker-process layer that does not exist yet.
 - Process-backed sandboxes are not attached to cgroups or watched by the
   resource watchdog (they get `rlimit` only).
 - `backend="microvm"` fails closed: the guest agent and vsock cell transport are

README.md

@@ -227,17 +227,39 @@ an execution cell and *not* a boundary against hostile Python, which is equally
 true of a thread and of a real sub-interpreter. What the old name obscured was
 the mechanism you should assume when reasoning about it:
 
-| | `thread` (today) | `subinterpreter` (reserved) |
+| | `thread` | `subinterpreter` |
 | --- | --- | --- |
 | Address space | shared with supervisor | shared with supervisor |
 | `sys.modules` | shared with supervisor | per-interpreter |
+| Import allow-list | thread-local bookkeeping | a property of the interpreter |
 | Boundary vs hostile Python | none | none |
 | GIL | shared | per-interpreter; irrelevant on free-threaded builds |
-
-Landing the real implementation (`concurrent.interpreters` on 3.14) is roadmap
-work; see [ROADMAP.md](ROADMAP.md) and the measured cost of a cell in
-[Performance snapshot](#performance-snapshot). Until then, use
-`backend="process"` for any guest you do not trust.
+| Requires | any supported Python | CPython 3.14+ |
+
+`backend="subinterpreter"` runs each guest in its own CPython interpreter via
+`concurrent.interpreters`. It needs CPython 3.14+ and **fails closed** below
+that rather than quietly handing back a thread, which isolates differently.
+It is not the default for that reason.
+
+Neither is a boundary against hostile Python: both share the supervisor's
+address space, `ctypes` imports cleanly inside a cell, and any C extension can
+reach the whole process. Use `backend="process"` for any guest you do not
+trust. What a cell buys over a thread is that one tenant's imports,
+monkey-patches and globals cannot be seen or clobbered by another.
+
+Cells are pooled and pre-warmed, because creating one costs 10-57 ms while
+dispatching onto a warm one costs 0.8 ms — see
+[Performance snapshot](#performance-snapshot). A released cell is *retired*
+rather than returned to the pool: an interpreter cannot be reset, so reusing
+one across tenants would carry the first tenant's globals into the second.
+
+One operational limit is worth knowing before you deploy it: **a running cell
+cannot be reclaimed.** `Interpreter.close()` refuses while the guest is
+executing and there is no `kill`, so a cell that overruns its deadline is
+*abandoned* — the sandbox raises, the pool stops using that cell, and its
+thread stays pinned until the process exits. If you need to survive runaway
+guests, run a pool of worker processes and treat the worker as the kill
+domain.
 
 ---
 

ROADMAP.md

@@ -8,7 +8,9 @@ normative statement.
 ## Delivered
 
 - **Backends** — `backend="thread"` (execution cell; a dedicated thread of the
-  supervisor process, previously spelled `subinterpreter`) and
+  supervisor process, previously spelled `subinterpreter`),
+  `backend="subinterpreter"` (real CPython sub-interpreter cells on 3.14+, with
+  a pre-warmed pool; an execution cell, not a boundary), and
   `backend="process"` (the boundary mode): a real separate-process boundary with
   `no_new_privs` + a seccomp deny-list, Landlock filesystem rules, Landlock
   TCP-egress rules (ABI ≥ 4), a coarse per-cgroup eBPF/LSM deny-mask, and
@@ -31,14 +33,17 @@ normative statement.
 
 ## Now / next
 
-- **A real `backend="subinterpreter"`** — the name is now free: the thread
-  runtime it used to label is called `backend="thread"`, and `subinterpreter`
-  resolves to it with a `DeprecationWarning` until the real thing lands. Build
-  it on `concurrent.interpreters` (3.14+, rather than the private
-  `_interpreters`, which heap-corrupts on realistic import surfaces — see
-  `scripts/cell_cost.py`). The boundary claim is unchanged: it is an execution
-  cell, not a boundary against hostile Python. What it adds over a thread is a
-  private `sys.modules` and a private set of globals per tenant.
+- **A kill domain for cells** — a running sub-interpreter cannot be reclaimed:
+  `close()` refuses while the guest is executing and there is no `kill`, so a
+  cell that overruns is abandoned and its thread is pinned for the life of the
+  process. The fix is not in-process. Add a layer of pre-forked worker
+  processes between the supervisor and the cells, size them by tenant, and make
+  the worker the unit that gets killed and replaced. This is what turns the cell
+  pool into something that survives a hostile-by-accident tenant.
+- **Per-cell resource accounting** — there is none today.
+  `sys.getallocatedblocks()` is process-global on both free-threaded and GIL
+  builds, so memory has to be capped at the worker/cgroup level rather than per
+  cell. Cells currently enforce a wall-time deadline and nothing else.
 - **Broker request execution** — the `request` op currently surfaces a
   `BrokerRequest` to the host but nothing executes it or returns a result. Add a
   request/response round-trip and a pluggable, capability-scoped handler so the