From f1e7f1e656b7c1a3d53488d82be86f6b438091da Mon Sep 17 00:00:00 2001 From: Claude Date: Wed, 16 Sep 2026 23:24:06 +0000 Subject: [PATCH 1/2] Add a real sub-interpreter backend with a pre-warmed cell pool backend="subinterpreter" now runs each guest in its own CPython interpreter via concurrent.interpreters, rather than naming a runtime it did not have. Each cell gets its own sys.modules and its own builtins, which is the point: the import allow-list stops being thread-local bookkeeping inside one shared interpreter and becomes a property of the interpreter, so one tenant's imports, monkey-patches and globals cannot be observed or clobbered by another. It is still not a boundary against hostile Python -- a cell shares the supervisor's address space and ctypes imports cleanly inside one -- and the docs say so in the same places they always have. Three measured properties of the CPython primitive shape the design: * Creating a cell costs 10-57ms and 3.5-13MiB depending on its import surface, against 0.8ms to dispatch onto a warm one. So cells are pooled and pre-warmed by CellPool, keyed by a CellSpec, and a request never pays for creation when a cell warmed the same way exists. * An interpreter cannot be reset. Releasing a cell therefore retires it and warms a replacement in the background; returning it to the pool would carry one tenant's globals into the next. * A running interpreter cannot be reclaimed. close() refuses while the guest is executing, and an async exception aimed at the thread does not reach it. A cell that overruns its deadline is abandoned, not killed: the sandbox raises WallTimeExceeded saying so, the pool counts it, and the thread stays pinned until the process exits. The API does not pretend otherwise -- kill() returns False rather than claiming a stop it cannot perform. Requires 3.14+ and fails closed below it with a diagnostic naming the alternatives, rather than degrading to the thread backend, which isolates differently. The private _interpreters on 3.12/3.13 is not used as a fallback: destroying interpreters that imported http.client or email.message aborts the process there, which is exactly the workload a pool generates. Two hazards found while building this, both fixed here rather than documented as quirks: * Messages cross as JSON in a single str. A cross-interpreter queue falls back to pickle for anything not natively shareable, which would have the supervisor unpickling bytes the guest produced -- the thing the process backend explicitly refuses to do. A str is natively shareable, so the fallback never engages. It also means a guest posting something unserialisable gets a TypeError at the call rather than an opaque NotShareableError from the runtime. * retire() asks the runtime whether the interpreter is running before closing it. Losing that race is not a catchable error: CPython aborts the process with "Py_EndInterpreter: not the last thread", which would take every other tenant down with the lost cell. The import hook also has to let CPython's own machinery import pickle and traceback. Blocking those does not produce a policy denial; it produces NotShareableError from two layers down. The allow-list gates guest imports; starving the interpreter's plumbing is not a security control. Co-Authored-By: Claude Opus 5 Claude-Session: https://claude.ai/code/session_01Fs1hTtmF4Hm9h617AG9Gse --- CHANGELOG.md | 20 +- README.md | 34 +- ROADMAP.md | 23 +- pyisolate/runtime/subinterpreter.py | 932 +++++++++++++++++++++++++++ pyisolate/supervisor.py | 129 +++- tests/test_subinterpreter_backend.py | 520 +++++++++++++++ tests/test_supervisor.py | 44 +- 7 files changed, 1641 insertions(+), 61 deletions(-) create mode 100644 pyisolate/runtime/subinterpreter.py create mode 100644 tests/test_subinterpreter_backend.py diff --git a/CHANGELOG.md b/CHANGELOG.md index bcc92e3..6b5c73a 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -8,6 +8,13 @@ guarantees; **no release should be treated as a hardened security boundary**. ## [Unreleased] ### Added +- `backend="subinterpreter"`: real CPython sub-interpreter cells on 3.14+, via + `concurrent.interpreters`, with a pre-warmed `CellPool`. Each guest gets its + own `sys.modules` and its own `builtins`, so the import allow-list is a + property of the interpreter rather than thread-local state. Still an + execution cell, not a boundary against hostile Python. Fails closed below + 3.14 rather than degrading to the thread backend. + - `backend="process"` boundary mode: a real separate-process boundary confined by `no_new_privs` + a seccomp deny-list, Landlock filesystem rules, Landlock TCP-egress rules (Landlock ABI ≥ 4), a coarse per-cgroup eBPF/LSM deny-mask, @@ -25,17 +32,20 @@ guarantees; **no release should be treated as a hardened security boundary**. ### Changed - `backend="subinterpreter"` is renamed to `backend="thread"`, which is what it - has always run. The old spelling still resolves and emits a - `DeprecationWarning`; it is reserved for a real CPython sub-interpreter - backend rather than kept as a permanent synonym, so callers who want the - thread runtime should pass `"thread"`. `DEPRECATED_BACKEND_ALIASES` is - exported alongside `SUPPORTED_BACKENDS`. + has always run, and the `subinterpreter` name now selects the real + sub-interpreter backend. `DEPRECATED_BACKEND_ALIASES` is exported alongside + `SUPPORTED_BACKENDS` and is currently empty. - Threat model and `SECURITY.md` reconciled with the real, backend-conditional boundary (the sub-interpreter backend is an execution cell, not a boundary against hostile Python). ### Known gaps - The broker `request` op is surfaced but not yet executed end-to-end. +- A running sub-interpreter cell cannot be reclaimed: one that overruns its + wall-time deadline is abandoned, and its thread stays pinned until the + process exits. Cells enforce a wall-time deadline and no other quota; + `sys.getallocatedblocks()` is process-global, so per-cell memory + accounting needs a worker-process layer that does not exist yet. - Process-backed sandboxes are not attached to cgroups or watched by the resource watchdog (they get `rlimit` only). - `backend="microvm"` fails closed: the guest agent and vsock cell transport are diff --git a/README.md b/README.md index 2ff9127..4c79331 100644 --- a/README.md +++ b/README.md @@ -227,17 +227,39 @@ an execution cell and *not* a boundary against hostile Python, which is equally true of a thread and of a real sub-interpreter. What the old name obscured was the mechanism you should assume when reasoning about it: -| | `thread` (today) | `subinterpreter` (reserved) | +| | `thread` | `subinterpreter` | | --- | --- | --- | | Address space | shared with supervisor | shared with supervisor | | `sys.modules` | shared with supervisor | per-interpreter | +| Import allow-list | thread-local bookkeeping | a property of the interpreter | | Boundary vs hostile Python | none | none | | GIL | shared | per-interpreter; irrelevant on free-threaded builds | - -Landing the real implementation (`concurrent.interpreters` on 3.14) is roadmap -work; see [ROADMAP.md](ROADMAP.md) and the measured cost of a cell in -[Performance snapshot](#performance-snapshot). Until then, use -`backend="process"` for any guest you do not trust. +| Requires | any supported Python | CPython 3.14+ | + +`backend="subinterpreter"` runs each guest in its own CPython interpreter via +`concurrent.interpreters`. It needs CPython 3.14+ and **fails closed** below +that rather than quietly handing back a thread, which isolates differently. +It is not the default for that reason. + +Neither is a boundary against hostile Python: both share the supervisor's +address space, `ctypes` imports cleanly inside a cell, and any C extension can +reach the whole process. Use `backend="process"` for any guest you do not +trust. What a cell buys over a thread is that one tenant's imports, +monkey-patches and globals cannot be seen or clobbered by another. + +Cells are pooled and pre-warmed, because creating one costs 10-57 ms while +dispatching onto a warm one costs 0.8 ms — see +[Performance snapshot](#performance-snapshot). A released cell is *retired* +rather than returned to the pool: an interpreter cannot be reset, so reusing +one across tenants would carry the first tenant's globals into the second. + +One operational limit is worth knowing before you deploy it: **a running cell +cannot be reclaimed.** `Interpreter.close()` refuses while the guest is +executing and there is no `kill`, so a cell that overruns its deadline is +*abandoned* — the sandbox raises, the pool stops using that cell, and its +thread stays pinned until the process exits. If you need to survive runaway +guests, run a pool of worker processes and treat the worker as the kill +domain. --- diff --git a/ROADMAP.md b/ROADMAP.md index a6f76f9..ef32644 100644 --- a/ROADMAP.md +++ b/ROADMAP.md @@ -8,7 +8,9 @@ normative statement. ## Delivered - **Backends** — `backend="thread"` (execution cell; a dedicated thread of the - supervisor process, previously spelled `subinterpreter`) and + supervisor process, previously spelled `subinterpreter`), + `backend="subinterpreter"` (real CPython sub-interpreter cells on 3.14+, with + a pre-warmed pool; an execution cell, not a boundary), and `backend="process"` (the boundary mode): a real separate-process boundary with `no_new_privs` + a seccomp deny-list, Landlock filesystem rules, Landlock TCP-egress rules (ABI ≥ 4), a coarse per-cgroup eBPF/LSM deny-mask, and @@ -31,14 +33,17 @@ normative statement. ## Now / next -- **A real `backend="subinterpreter"`** — the name is now free: the thread - runtime it used to label is called `backend="thread"`, and `subinterpreter` - resolves to it with a `DeprecationWarning` until the real thing lands. Build - it on `concurrent.interpreters` (3.14+, rather than the private - `_interpreters`, which heap-corrupts on realistic import surfaces — see - `scripts/cell_cost.py`). The boundary claim is unchanged: it is an execution - cell, not a boundary against hostile Python. What it adds over a thread is a - private `sys.modules` and a private set of globals per tenant. +- **A kill domain for cells** — a running sub-interpreter cannot be reclaimed: + `close()` refuses while the guest is executing and there is no `kill`, so a + cell that overruns is abandoned and its thread is pinned for the life of the + process. The fix is not in-process. Add a layer of pre-forked worker + processes between the supervisor and the cells, size them by tenant, and make + the worker the unit that gets killed and replaced. This is what turns the cell + pool into something that survives a hostile-by-accident tenant. +- **Per-cell resource accounting** — there is none today. + `sys.getallocatedblocks()` is process-global on both free-threaded and GIL + builds, so memory has to be capped at the worker/cgroup level rather than per + cell. Cells currently enforce a wall-time deadline and nothing else. - **Broker request execution** — the `request` op currently surfaces a `BrokerRequest` to the host but nothing executes it or returns a result. Add a request/response round-trip and a pluggable, capability-scoped handler so the diff --git a/pyisolate/runtime/subinterpreter.py b/pyisolate/runtime/subinterpreter.py new file mode 100644 index 0000000..5e2a425 --- /dev/null +++ b/pyisolate/runtime/subinterpreter.py @@ -0,0 +1,932 @@ +"""Real CPython sub-interpreter cells -- the ``backend="subinterpreter"`` runtime. + +This is the backend the name always described. Each guest runs in its own +CPython interpreter created through :mod:`concurrent.interpreters`, with its own +``sys.modules``, its own ``builtins``, and its own module state. Compared with +``backend="thread"``, that is the whole point: the import allow-list stops being +thread-local bookkeeping inside one shared interpreter and becomes a property of +the interpreter itself, so one tenant's imports, monkey-patches and globals +cannot be observed or clobbered by another. + +It is still **not** a security boundary against hostile Python. A cell shares the +supervisor's address space; ``ctypes`` imports cleanly inside one, and any C +extension can reach the whole process. Use ``backend="process"`` for code you do +not trust. What a cell buys is fault and namespace isolation between tenants +whose code is trusted but independent, plus parallelism on a GIL build. + +Three properties of the underlying CPython primitive shape everything here, all +of them measured by ``scripts/cell_cost.py``: + +**Creating a cell is expensive.** A sub-interpreter re-imports every module it +uses with no copy-on-write sharing, so 10 ms and 3.5 MiB bare grows to 57 ms and +13 MiB once a realistic module surface is imported. Dispatch onto an +already-warm interpreter is 0.8 ms. So cells are pooled and pre-warmed by +:class:`CellPool`, and a cell is never created on a request path if a warm one +matching the spec exists. + +**A cell cannot be reset.** There is no API to return an interpreter to a +pristine state, and reusing one across tenants would leak the first tenant's +globals into the second. So the lifecycle is lease -> run -> *retire*: a +released cell is destroyed and the pool warms a replacement in the background. + +**A running cell cannot be reclaimed.** ``Interpreter.close()`` raises +``InterpreterError: interpreter running``, and an async exception aimed at the +thread does not reach the interpreter actually executing. There is no +``interp.kill()``. A cell that overruns its deadline is therefore *abandoned*, +not killed: :meth:`Cell.abandon` marks it unusable and the thread stays pinned +until the process exits. The only reclaim that actually works is at the process +level, which is why a deployment that must survive runaway guests runs a pool of +worker processes and treats the worker as the kill domain. +""" + +from __future__ import annotations + +import json +import logging +import sys +import threading +import time +from dataclasses import dataclass +from enum import Enum +from typing import Any, Callable, Iterable, Optional + +from .. import errors + +logger = logging.getLogger(__name__) + +#: ``concurrent.interpreters`` (PEP 734) is the supported API. The private +#: ``_interpreters`` on 3.12/3.13 is not a fallback: destroying interpreters +#: that have imported a realistic module surface (``http.client``, +#: ``email.message``) aborts the process with ``munmap_chunk(): invalid +#: pointer`` there, which is exactly the workload a cell pool generates. +MIN_PYTHON = (3, 14) + +try: # pragma: no cover - selected by the running interpreter + import concurrent.interpreters as _interpreters + + _IMPORT_ERROR: Optional[str] = None +except ImportError as exc: # pragma: no cover - 3.11/3.12/3.13 + _interpreters = None # type: ignore[assignment] + _IMPORT_ERROR = str(exc) + + +def is_available() -> bool: + """Whether this build can run sub-interpreter cells.""" + return _interpreters is not None and sys.version_info[:2] >= MIN_PYTHON + + +def require_available() -> None: + """Fail closed, naming what is missing, rather than degrading to a thread. + + Silently falling back to ``backend="thread"`` would hand the caller a + different isolation model than the one they asked for, which is the mistake + the backend rename was meant to end. + """ + if is_available(): + return + running = f"{sys.version_info[0]}.{sys.version_info[1]}" + needed = f"{MIN_PYTHON[0]}.{MIN_PYTHON[1]}" + detail = f" ({_IMPORT_ERROR})" if _IMPORT_ERROR else "" + raise errors.SandboxError( + f"backend='subinterpreter' needs CPython {needed}+ for " + f"concurrent.interpreters; this is {running}{detail}. The private " + "_interpreters module on 3.12/3.13 is not used as a fallback because " + "destroying interpreters that imported a realistic module surface " + "aborts the process there. Use backend='thread' for an in-process " + "execution cell, or backend='process' for a real boundary." + ) + + +# --- what a warm cell contains -------------------------------------------- + + +@dataclass(frozen=True) +class CellSpec: + """The shape of a pre-warmed cell. + + Two cells are interchangeable only if they were warmed identically, so this + is the pool's key. It is frozen and hashable for that reason. + + ``preimport`` is the part of the cost worth paying ahead of a request: those + modules are imported while the cell is warming rather than when a guest + first touches them. It must be a subset of ``allowed_imports`` -- warming a + cell with a module its policy forbids would import it anyway. + """ + + allowed_imports: frozenset[str] = frozenset() + preimport: tuple[str, ...] = () + + def __post_init__(self) -> None: + forbidden = [m for m in self.preimport if not self.allows(m)] + if forbidden: + raise ValueError( + f"preimport modules not in allowed_imports: {sorted(forbidden)}" + ) + + @classmethod + def build( + cls, + allowed_imports: Optional[Iterable[str]] = None, + preimport: Optional[Iterable[str]] = None, + ) -> "CellSpec": + allowed = frozenset(allowed_imports or ()) + # Default to pre-importing everything the policy allows: the import is + # going to happen anyway, and paying for it off the request path is the + # only reason a pool is worth having. + chosen = ( + tuple(sorted(preimport)) + if preimport is not None + else tuple(sorted(allowed)) + ) + return cls(allowed_imports=allowed, preimport=chosen) + + def allows(self, name: str) -> bool: + """Mirror of the thread backend's allow-list semantics.""" + if name in self.allowed_imports: + return True + # ``import package.child`` imports ``package`` first, so a parent is + # allowed exactly when something more specific under it is. + return any(a.startswith(f"{name}.") for a in self.allowed_imports) + + +#: CPython's own cross-interpreter machinery imports these from inside the cell. +#: A ``Queue`` falls back to ``pickle`` for anything not natively shareable, and +#: ``Interpreter.exec`` uses ``pickle``/``traceback`` to carry a guest exception +#: back to the caller. Those imports go through whatever ``__import__`` the cell +#: currently has, so an allow-list that does not include them does not produce a +#: policy denial -- it produces ``NotShareableError`` from the runtime, with the +#: real cause two layers down. The allow-list gates *guest* imports; starving the +#: interpreter's own plumbing is not a security control. +_RUNTIME_IMPORTS = ("pickle", "traceback", "copyreg") + +#: Installed in every cell before any guest code runs. It executes inside the +#: cell's own interpreter, so it rebinds *that* interpreter's ``builtins`` -- +#: the supervisor's is untouched, which is what makes the allow-list a property +#: of the interpreter instead of thread-local state. +#: +#: Every message is JSON-encoded here, in the cell, and crosses as one ``str``. +#: A ``str`` is natively shareable, so the queue's pickle fallback never +#: engages and the supervisor never unpickles bytes the guest produced. That is +#: the same rule the process backend applies to values crossing its boundary, +#: and it is enforced at the point of the call: a guest that posts something +#: unserialisable gets a ``TypeError`` from ``post`` rather than an opaque +#: sharing error from the runtime. +_BOOTSTRAP = """ +import builtins as _pyi_builtins +import json as _pyi_json + +_PYI_ALLOWED = frozenset(__pyi_allowed__) +_PYI_RUNTIME = frozenset(__pyi_runtime_imports__) +_pyi_real_import = _pyi_builtins.__import__ +_pyi_outbox = __pyi_outbox__ + + +def _pyi_is_allowed(name): + if name in _PYI_ALLOWED or name in _PYI_RUNTIME: + return True + prefix = name + "." + for allowed in _PYI_ALLOWED: + if allowed.startswith(prefix): + return True + return False + + +def _pyi_import(name, globals=None, locals=None, fromlist=(), level=0): + requested = name + if level: + package = globals.get("__package__") if isinstance(globals, dict) else None + base = (package or "") + if level > 1: + base = base.rsplit(".", level - 1)[0] + requested = base + "." + name if name else base + if not _pyi_is_allowed(requested): + raise ImportError("import of %r is not permitted by policy" % (requested,)) + return _pyi_real_import(name, globals, locals, fromlist, level) + + +def _pyi_emit(kind, payload): + _pyi_outbox.put(_pyi_json.dumps([kind, payload])) + + +def post(value): + # Send a value to the supervisor. Must be JSON-serialisable. + _pyi_emit("post", value) + + +def log(level, message, **fields): + _pyi_emit("log", {"level": str(level), "message": str(message), "fields": fields}) + + +def metric(name, value): + _pyi_emit("metric", {"name": str(name), "value": value}) + + +def request(capability, *args, **kwargs): + # Ask the broker to perform a privileged operation for this cell. + _pyi_emit( + "request", + {"capability": str(capability), "args": list(args), "kwargs": kwargs}, + ) + + +_pyi_builtins.post = post +_pyi_builtins.log = log +_pyi_builtins.metric = metric +_pyi_builtins.request = request +# Installed last: everything above needs a working import. +_pyi_builtins.__import__ = _pyi_import +""" + + +class CellState(str, Enum): + """Lifecycle of one cell. There is no path back from ABANDONED.""" + + WARMING = "warming" + IDLE = "idle" + RUNNING = "running" + RETIRED = "retired" + ABANDONED = "abandoned" + + +class Cell: + """One CPython sub-interpreter plus the queue it reports through.""" + + __slots__ = ("_interp", "_outbox", "_spec", "_state", "_lock", "_created_at") + + def __init__(self, spec: CellSpec) -> None: + require_available() + self._spec = spec + self._state = CellState.WARMING + self._lock = threading.Lock() + self._created_at = time.monotonic() + self._interp = _interpreters.create() + self._outbox = _interpreters.create_queue() + self._bootstrap() + + def _bootstrap(self) -> None: + self._interp.prepare_main( + __pyi_outbox__=self._outbox, + __pyi_allowed__=tuple(sorted(self._spec.allowed_imports)), + __pyi_runtime_imports__=_RUNTIME_IMPORTS, + ) + self._interp.exec(_BOOTSTRAP) + for module in self._spec.preimport: + # Warming failures are not fatal: a module the host cannot import + # into a sub-interpreter (numpy and cryptography both refuse) must + # surface when the guest asks for it, not turn the pool into a + # source of startup errors. + try: + self._interp.exec(f"import {module}") + except Exception as exc: # pragma: no cover - depends on host + logger.warning( + "cell pre-import of %r failed; it will fail for the guest " + "too: %s", + module, + exc, + ) + self._state = CellState.IDLE + + # --- introspection --- + + @property + def spec(self) -> CellSpec: + return self._spec + + @property + def state(self) -> CellState: + return self._state + + @property + def id(self) -> int: + return self._interp.id + + def is_usable(self) -> bool: + return self._state in (CellState.IDLE, CellState.RUNNING) + + def _is_running(self) -> bool: + """Whether the runtime considers the interpreter to be executing.""" + try: + return bool(self._interp.is_running()) + except Exception: # pragma: no cover - a destroyed interpreter + return False + + # --- the cell ABI --- + + def exec(self, source: str) -> None: + """Run *source* in the cell. Raises the guest's exception on failure.""" + self._enter() + try: + self._interp.exec(source) + finally: + self._leave() + + def call(self, func: Callable[..., Any], *args: Any, **kwargs: Any) -> Any: + """Call *func* inside the cell and return its result.""" + self._enter() + try: + return self._interp.call(func, *args, **kwargs) + finally: + self._leave() + + def exec_in_thread(self, source: str) -> threading.Thread: + """Run *source* on a thread of its own so the caller can time it out. + + The returned thread is the only handle on that work. If it overruns, + there is nothing to cancel -- see :meth:`abandon`. + """ + self._enter() + + def _run() -> None: + try: + self._interp.exec(source) + except BaseException: # pragma: no cover - surfaced via drain + logger.debug("cell %s raised in exec_in_thread", self.id, exc_info=True) + finally: + self._leave() + + thread = threading.Thread(target=_run, name=f"pyisolate-cell-{self.id}") + thread.daemon = True + thread.start() + return thread + + def drain(self) -> list[tuple[str, Any]]: + """Return every ``(kind, payload)`` the cell has emitted so far.""" + messages: list[tuple[str, Any]] = [] + while True: + try: + raw = self._outbox.get_nowait() + except Exception: + return messages + decoded = self._decode(raw) + if decoded is not None: + messages.append(decoded) + + @staticmethod + def _decode(raw: Any) -> Optional[tuple[str, Any]]: + """Parse one cell message, dropping anything malformed. + + Only JSON produced by the bootstrap is expected. Guest code can reach + the outbox and put whatever it likes on it, so this validates rather + than trusts: a malformed frame is logged and discarded instead of + propagating an arbitrary object into the supervisor. + """ + if not isinstance(raw, str): + logger.warning("discarding non-string cell message %r", type(raw)) + return None + try: + parsed = json.loads(raw) + except (TypeError, ValueError): + logger.warning("discarding unparseable cell message") + return None + if ( + not isinstance(parsed, list) + or len(parsed) != 2 + or not isinstance(parsed[0], str) + ): + logger.warning("discarding malformed cell message") + return None + return parsed[0], parsed[1] + + # --- lifecycle --- + + def _enter(self) -> None: + with self._lock: + if self._state is CellState.ABANDONED: + raise errors.SandboxError("cell was abandoned and cannot be reused") + if self._state is CellState.RETIRED: + raise errors.SandboxError("cell was retired and cannot be reused") + self._state = CellState.RUNNING + + def _leave(self) -> None: + with self._lock: + if self._state is CellState.RUNNING: + self._state = CellState.IDLE + + def retire(self) -> bool: + """Destroy the interpreter. Returns whether it actually went away. + + This is the only way to make a cell safe to hand to another tenant: + there is no reset, so a cell that has run one tenant's code is spent. + """ + with self._lock: + if self._state in (CellState.RETIRED, CellState.ABANDONED): + return self._state is CellState.RETIRED + # Ask the runtime, not our own state flag. ``exec_in_thread`` marks the + # cell RUNNING before its thread has actually entered the interpreter, + # and closing one that a thread is inside is not a catchable error: + # CPython aborts the whole process with "Py_EndInterpreter: not the + # last thread". Refusing here keeps a lost cell from taking every other + # tenant in the process down with it. + if self._is_running(): + logger.warning("cell %s is still running; cannot retire it", self.id) + with self._lock: + self._state = CellState.ABANDONED + return False + try: + self._interp.close() + except Exception as exc: + # Lost the race anyway: the guest re-entered between the check and + # the close. Record it rather than pretending the cell closed. + logger.warning("cell %s could not be retired: %s", self.id, exc) + with self._lock: + self._state = CellState.ABANDONED + return False + with self._lock: + self._state = CellState.RETIRED + return True + + def abandon(self, reason: str) -> None: + """Mark a cell unusable without claiming it was reclaimed. + + Called when a guest overruns its deadline. The interpreter keeps + running and its thread stays pinned for the life of the process; the + honest accounting is that the cell is lost, not killed. + """ + with self._lock: + self._state = CellState.ABANDONED + logger.warning("cell %s abandoned: %s", self.id, reason) + + def kill(self, timeout: float = 0.2) -> bool: + """Always ``False`` while the cell is running. CPython has no kill. + + Kept so the backend answers the same question as the others rather than + raising ``AttributeError``, and so callers see the ``False`` and + escalate to the process level instead of assuming a stopped guest. + """ + del timeout + if self._state is CellState.RUNNING: + return False + return self.retire() + + +# --- the pool ------------------------------------------------------------- + + +@dataclass +class PoolStats: + """Counters that make the pool's behaviour legible in production.""" + + created: int = 0 + reused: int = 0 + retired: int = 0 + abandoned: int = 0 + warm: int = 0 + + def as_dict(self) -> dict[str, int]: + return { + "created": self.created, + "reused": self.reused, + "retired": self.retired, + "abandoned": self.abandoned, + "warm": self.warm, + } + + +class CellPool: + """Keeps pre-warmed cells so a request pays 0.8 ms rather than 10-57 ms. + + Cells are keyed by :class:`CellSpec`, because a warm cell is only useful to + a guest that wants exactly the module surface it was warmed with. Handing + back a cell warmed for a different policy would either import the missing + modules on the request path -- losing the point -- or run the guest against + an allow-list that is not its own. + """ + + def __init__(self, warm_per_spec: int = 1, max_warm: int = 32) -> None: + if warm_per_spec < 0: + raise ValueError("warm_per_spec must be >= 0") + if max_warm < 0: + raise ValueError("max_warm must be >= 0") + self._warm_per_spec = warm_per_spec + self._max_warm = max_warm + self._warm: dict[CellSpec, list[Cell]] = {} + self._lock = threading.Lock() + self._stats = PoolStats() + self._closed = False + # Warming runs on background threads. They must be tracked, because + # creating or destroying a sub-interpreter while the runtime is + # finalising is not safe: close() joins them before retiring anything. + self._warming: set[threading.Thread] = set() + + # --- lease/return --- + + def acquire(self, spec: CellSpec) -> Cell: + """Return a cell matching *spec*, warm if one is available.""" + with self._lock: + if self._closed: + raise errors.SandboxError("cell pool is closed") + waiting = self._warm.get(spec) + while waiting: + cell = waiting.pop() + if cell.is_usable(): + self._stats.reused += 1 + self._stats.warm = self._count_warm_locked() + return cell + return self._create(spec) + + def release(self, cell: Cell) -> None: + """Retire *cell* and warm a replacement. + + Deliberately not a return-to-pool: an interpreter cannot be reset, so + reusing one across tenants would carry the first tenant's globals into + the second. + """ + retired = cell.retire() + with self._lock: + if retired: + self._stats.retired += 1 + else: + self._stats.abandoned += 1 + closed = self._closed + if not closed and retired: + self._refill(cell.spec) + + def abandon(self, cell: Cell, reason: str) -> None: + """Record a cell that overran and cannot be taken back.""" + cell.abandon(reason) + with self._lock: + self._stats.abandoned += 1 + + # --- warming --- + + def prewarm(self, spec: CellSpec, count: Optional[int] = None) -> int: + """Create warm cells for *spec* up front. Returns how many were added.""" + target = self._warm_per_spec if count is None else count + added = 0 + for _ in range(target): + if not self._has_warm_capacity(): + break + cell = self._create(spec) + with self._lock: + if self._closed: + # Closed while this cell was being built. Retire it here + # rather than parking it in a pool nobody will drain. + closed = True + else: + self._warm.setdefault(spec, []).append(cell) + self._stats.warm = self._count_warm_locked() + closed = False + if closed: + cell.retire() + break + added += 1 + return added + + def _refill(self, spec: CellSpec) -> None: + """Warm a replacement off the request path.""" + if self._warm_per_spec <= 0: + return + + def _warm() -> None: + try: + self.prewarm(spec, 1) + except Exception: # pragma: no cover - best effort + logger.warning("failed to warm a replacement cell", exc_info=True) + finally: + with self._lock: + self._warming.discard(threading.current_thread()) + + thread = threading.Thread(target=_warm, name="pyisolate-cell-warm") + # Daemon so a wedged warm never blocks interpreter exit; close() still + # joins it, so the normal path does not race teardown. + thread.daemon = True + with self._lock: + if self._closed: + return + self._warming.add(thread) + thread.start() + + def _has_warm_capacity(self) -> bool: + with self._lock: + if self._closed: + return False + return self._count_warm_locked() < self._max_warm + + def _count_warm_locked(self) -> int: + return sum(len(cells) for cells in self._warm.values()) + + def _create(self, spec: CellSpec) -> Cell: + cell = Cell(spec) + with self._lock: + self._stats.created += 1 + return cell + + # --- teardown --- + + def stats(self) -> dict[str, int]: + with self._lock: + self._stats.warm = self._count_warm_locked() + return self._stats.as_dict() + + def close(self, timeout: float = 5.0) -> None: + """Retire every warm cell. Abandoned ones cannot be reclaimed. + + In-flight warm threads are joined first. Creating an interpreter + concurrently with the runtime tearing one down is not safe, so closing + without waiting turns an orderly shutdown into a race. + """ + with self._lock: + self._closed = True + warming = list(self._warming) + deadline = time.monotonic() + timeout + for thread in warming: + thread.join(max(0.0, deadline - time.monotonic())) + if thread.is_alive(): # pragma: no cover - needs a wedged warm + logger.warning( + "warm thread %s did not finish before close", thread.name + ) + with self._lock: + warm = [cell for cells in self._warm.values() for cell in cells] + self._warm.clear() + self._warming.clear() + for cell in warm: + if not cell.retire(): # pragma: no cover - needs a stuck cell + with self._lock: + self._stats.abandoned += 1 + + def __enter__(self) -> "CellPool": + return self + + def __exit__(self, *exc: object) -> None: + self.close() + + +# --- the sandbox handle --------------------------------------------------- + + +@dataclass +class _Deadline: + wall_time_ms: Optional[int] = None + + def seconds(self) -> Optional[float]: + if self.wall_time_ms is None: + return None + return self.wall_time_ms / 1000.0 + + +class SubinterpreterSandbox: + """Sandbox handle over a pooled cell, exposing the minimal cell ABI.""" + + def __init__( + self, + name: str, + *, + pool: CellPool, + spec: CellSpec, + wall_time_ms: Optional[int] = None, + ) -> None: + require_available() + self.name = name + self._pool = pool + self._spec = spec + self._deadline = _Deadline(wall_time_ms) + self._cell: Optional[Cell] = pool.acquire(spec) + self._posted: list[Any] = [] + self._logs: list[Any] = [] + self._metrics: list[Any] = [] + self._requests: list[Any] = [] + self._lock = threading.Lock() + self._backend = "subinterpreter" + # Read directly by the Sandbox handle. A cell is not in a cgroup of its + # own -- it shares the supervisor's process -- and its quotas are the + # wall-time deadline only, which is what these report. + self._cgroup_path: Optional[str] = None + self._quarantine_reason: Optional[str] = None + self.termination_reason: Optional[str] = None + self.quota_enforcement = "wall_time_only" + + # --- ABI --- + + def exec(self, src: str) -> None: + cell = self._require_cell() + timeout = self._deadline.seconds() + if timeout is None: + cell.exec(src) + self._collect(cell) + return + thread = cell.exec_in_thread(src) + thread.join(timeout) + if thread.is_alive(): + # Nothing can stop it. Give up the cell and say so plainly. + self._pool.abandon(cell, f"exec exceeded {self._deadline.wall_time_ms}ms") + self._cell = None + raise errors.WallTimeExceeded( + f"sandbox '{self.name}' exceeded " + f"{self._deadline.wall_time_ms}ms; the cell was abandoned " + "because a running sub-interpreter cannot be reclaimed" + ) + self._collect(cell) + + def call( + self, + func: str, + *args: Any, + timeout: Optional[float] = None, + **kwargs: Any, + ) -> Any: + """Call a dotted function inside the cell and return its result. + + Takes a dotted *name*, matching the other backends, rather than a + callable. Resolving it inside the cell keeps the host from pickling one + of its own objects into the guest, and routes the lookup through the + cell's guarded ``__import__`` so the allow-list applies to it. + """ + del timeout # honoured by exec's deadline, not per-call + cell = self._require_cell() + if not isinstance(func, str) or not func: + raise TypeError("func must be a dotted name") + module, _, attr = func.rpartition(".") + if not module or not attr: + raise ValueError(f"expected a dotted name, got {func!r}") + # Reuse the decoder the bootstrap already bound. Importing json here + # would go through the guarded __import__ and be denied unless the + # policy happened to allow it, which has nothing to do with the call. + payload = json.dumps({"args": list(args), "kwargs": kwargs}) + cell.exec( + f"_pyi_call = _pyi_json.loads({payload!r})\n" + f"_pyi_mod = __import__({module!r}, fromlist=[{attr!r}])\n" + f"post(getattr(_pyi_mod, {attr!r})" + "(*_pyi_call['args'], **_pyi_call['kwargs']))\n" + ) + self._collect(cell) + with self._lock: + if not self._posted: + raise errors.SandboxError(f"call to {func!r} returned no result") + return self._posted.pop() + + # --- surface the Sandbox handle delegates to --- + + def cancel(self, timeout: float = 0.2) -> bool: + """Same answer as :meth:`kill`: there is nothing else to try.""" + return self.kill(timeout) + + def reap(self) -> bool: + self.close() + return True + + def quarantine(self, reason: str = "manual quarantine") -> None: + cell = self._cell + self._quarantine_reason = reason + if cell is not None: + self._pool.abandon(cell, reason) + self._cell = None + + def snapshot(self) -> dict[str, Any]: + """Serializable state for checkpointing. + + Deliberately excludes guest state: an interpreter's contents cannot be + captured or restored, so a checkpoint of a cell is its configuration, + not its memory. + """ + return { + "name": self.name, + "backend": self._backend, + "allowed_imports": sorted(self._spec.allowed_imports), + "wall_time_ms": self._deadline.wall_time_ms, + } + + def reset_config(self) -> dict[str, Any]: + raise NotImplementedError( + "a sub-interpreter cannot be reset to a pristine state, so a cell " + "is never reused across tenants. Close this sandbox and spawn " + "another; the pool keeps a warm cell so that costs ~1ms." + ) + + def reset(self, *args: Any, **kwargs: Any) -> None: + self.reset_config() + + def enable_tracing(self) -> None: + raise NotImplementedError( + "syscall tracing is a process-backend feature; a cell shares the " + "supervisor's process and has no syscall boundary to trace" + ) + + def get_syscall_log(self) -> list[str]: + return [] + + def get_denial_events(self) -> list[dict[str, str]]: + """Import denials are raised into the guest, not collected here.""" + return [] + + def profile(self) -> dict[str, Any]: + return self.stats() + + def recv(self, timeout: Optional[float] = None) -> Any: + with self._lock: + if self._posted: + return self._posted.pop(0) + cell = self._require_cell() + deadline = time.monotonic() + (timeout if timeout is not None else 0.0) + while True: + self._collect(cell) + with self._lock: + if self._posted: + return self._posted.pop(0) + if timeout is None or time.monotonic() >= deadline: + raise errors.TimeoutError(f"no message from sandbox '{self.name}'") + time.sleep(0.001) + + # --- supervisor surface --- + + def is_alive(self) -> bool: + cell = self._cell + return cell is not None and cell.is_usable() + + def kill(self, timeout: float = 0.2) -> bool: + cell = self._cell + if cell is None: + return True + return cell.kill(timeout) + + def stop(self, timeout: float = 0.2) -> None: + self.close(timeout) + + def close(self, timeout: float = 0.2) -> None: + del timeout + cell = self._cell + self._cell = None + if cell is not None: + self._pool.release(cell) + + def stats(self) -> dict[str, Any]: + cell = self._cell + return { + "backend": self._backend, + "cell_id": None if cell is None else cell.id, + "cell_state": None if cell is None else cell.state.value, + "posted": len(self._posted), + "requests": len(self._requests), + "pool": self._pool.stats(), + } + + def get_broker_requests(self) -> list[Any]: + """Requests the guest raised, for the supervisor's broker to mediate.""" + with self._lock: + return list(self._requests) + + def __enter__(self) -> "SubinterpreterSandbox": + return self + + def __exit__(self, *exc: object) -> None: + self.close() + + # --- internals --- + + def _require_cell(self) -> Cell: + cell = self._cell + if cell is None or not cell.is_usable(): + raise errors.SandboxError(f"sandbox '{self.name}' has no usable cell") + return cell + + def _collect(self, cell: Cell) -> None: + for kind, payload in cell.drain(): + with self._lock: + if kind == "post": + self._posted.append(payload) + elif kind == "log": + self._logs.append(payload) + elif kind == "metric": + self._metrics.append(payload) + elif kind == "request": + self._requests.append(payload) + else: + logger.warning("unknown cell message kind %r", kind) + + +#: Process-wide pool, created lazily so importing this module on a build +#: without sub-interpreters stays free. +_default_pool: Optional[CellPool] = None +_default_pool_lock = threading.Lock() + + +def default_pool() -> CellPool: + global _default_pool + with _default_pool_lock: + if _default_pool is None: + require_available() + _default_pool = CellPool() + return _default_pool + + +def reset_default_pool() -> None: + """Drop the process-wide pool. Used by tests and by supervisor shutdown.""" + global _default_pool + with _default_pool_lock: + pool, _default_pool = _default_pool, None + if pool is not None: + pool.close() + + +__all__ = [ + "MIN_PYTHON", + "Cell", + "CellPool", + "CellSpec", + "CellState", + "PoolStats", + "SubinterpreterSandbox", + "default_pool", + "is_available", + "require_available", + "reset_default_pool", +] diff --git a/pyisolate/supervisor.py b/pyisolate/supervisor.py index 3d4c95d..013deb3 100644 --- a/pyisolate/supervisor.py +++ b/pyisolate/supervisor.py @@ -26,6 +26,7 @@ from .observability.trace import Tracer from .policy import resolve_policy from .runtime import microvm as _microvm +from .runtime import subinterpreter from .runtime.process_backend import ProcessSandbox from .runtime.protocol import CapabilityHandle, ControlRequest from .runtime.thread import SandboxThread @@ -43,41 +44,39 @@ DEFAULT_NAME_PATTERN = re.compile(r"^[A-Za-z0-9_-]+$") NAME_PATTERN = DEFAULT_NAME_PATTERN -BackendMode = Literal["thread", "process", "microvm"] +BackendMode = Literal["thread", "subinterpreter", "process", "microvm"] DEFAULT_BACKEND: BackendMode = "thread" SUPPORTED_BACKENDS: tuple[BackendMode, ...] = ( "thread", + "subinterpreter", "process", "microvm", ) -IMPLEMENTED_BACKENDS: tuple[BackendMode, ...] = ("thread", "process") - -#: ``backend="subinterpreter"`` never selected a CPython sub-interpreter. It -#: selected -- and for now still selects -- the thread runtime in -#: :mod:`pyisolate.runtime.thread`, which ``exec``s guest source in a -#: ``threading.Thread`` of this process against a restricted ``__builtins__`` -#: mapping. Naming a backend for an implementation it does not have makes the -#: mechanism impossible to reason about from the API: a reader has to know that -#: ``sys.modules`` is shared, not per-interpreter, to judge what isolation they -#: are getting. -#: -#: So the runtime gets its real name, and the old one keeps working with a -#: warning. The alias is not a permanent synonym: when the real sub-interpreter -#: backend lands, ``"subinterpreter"`` is reassigned to *it*, which is why the -#: warning tells callers who want today's behaviour to say ``"thread"``. -DEPRECATED_BACKEND_ALIASES: dict[str, BackendMode] = {"subinterpreter": "thread"} +IMPLEMENTED_BACKENDS: tuple[BackendMode, ...] = ( + "thread", + "subinterpreter", + "process", +) + +#: ``"subinterpreter"`` used to be an alias for the thread runtime, which is +#: what it actually ran. It now names :mod:`pyisolate.runtime.subinterpreter`, +#: the real thing, so the alias is gone: the previous release warned that this +#: reassignment was coming and told callers who wanted the thread runtime to say +#: ``"thread"``. +DEPRECATED_BACKEND_ALIASES: dict[str, BackendMode] = {} + +#: ``"subinterpreter"`` stays the default only where it can actually run. It +#: needs CPython 3.14+, so making it the default would break every 3.11-3.13 +#: caller; the backend fails closed with a diagnostic instead of degrading to a +#: thread, and callers opt in. +SUBINTERPRETER_MIN_PYTHON = subinterpreter.MIN_PYTHON def _normalize_backend(backend: str) -> BackendMode: alias = DEPRECATED_BACKEND_ALIASES.get(backend) - if alias is not None: + if alias is not None: # pragma: no cover - no aliases at present warnings.warn( - f"backend={backend!r} is deprecated and currently selects " - f"backend={alias!r}, which runs the guest in a thread of this " - "process rather than in a CPython sub-interpreter. Pass " - f"backend={alias!r} to keep this behaviour; the " - f"{backend!r} name will be reassigned to a real sub-interpreter " - "backend.", + f"backend={backend!r} is deprecated; use backend={alias!r}.", DeprecationWarning, stacklevel=3, ) @@ -123,19 +122,29 @@ def _require_implemented_backend(backend: BackendMode) -> None: ) +#: Any of the three runtimes a :class:`Sandbox` can wrap. They are unrelated +#: classes that implement the same cell ABI rather than a shared base, so the +#: union is the type. +BackendSandbox = Union[ + "SandboxThread", "ProcessSandbox", "subinterpreter.SubinterpreterSandbox" +] + + class Sandbox: """Handle to a sandbox. - Wraps either a :class:`~pyisolate.runtime.thread.SandboxThread` - (``backend="thread"``) or a + Wraps a :class:`~pyisolate.runtime.thread.SandboxThread` + (``backend="thread"``), a + :class:`~pyisolate.runtime.subinterpreter.SubinterpreterSandbox` + (``backend="subinterpreter"``), or a :class:`~pyisolate.runtime.process_backend.ProcessSandbox` - (``backend="process"``); both expose the same cell-ABI surface this handle - delegates to. + (``backend="process"``); all three expose the same cell-ABI surface this + handle delegates to. """ def __init__( self, - thread: "Union[SandboxThread, ProcessSandbox]", + thread: "BackendSandbox", supervisor: "Supervisor", ): self._thread = thread @@ -258,6 +267,13 @@ def __init__( # SandboxThread instances, so the watchdog/warm-pool/cgroup machinery # that iterates ``_sandboxes`` must not see them. self._process_sandboxes: Dict[str, ProcessSandbox] = {} + # Sub-interpreter cells get their own registry for the same reason: they + # are not SandboxThread instances, so the watchdog and warm-pool + # machinery that walks ``_sandboxes`` must not see them either. + self._cell_sandboxes: Dict[str, "subinterpreter.SubinterpreterSandbox"] = {} + # Created on first use so that importing pyisolate on a build without + # concurrent.interpreters costs nothing. + self._pool: Optional["subinterpreter.CellPool"] = None self._lock = threading.Lock() self._alerts = AlertManager() self._tracer = Tracer() @@ -405,6 +421,13 @@ def spawn( imports.update(allowed_imports) allowed_imports = list(imports) + if backend == "subinterpreter": + return self._spawn_subinterpreter( + name, + allowed_imports=allowed_imports, + wall_time_ms=wall_time_ms, + ) + if backend == "process": return self._spawn_process( name, @@ -558,6 +581,42 @@ def _apply_kernel_policy(self, cg_path: Any, policy: Any) -> None: raise logger.debug("sandbox_policy map update skipped for %s", cg_path) + def _spawn_subinterpreter( + self, + name: str, + *, + allowed_imports: Optional[list[str]] = None, + wall_time_ms: Optional[int] = None, + ) -> Sandbox: + """Spawn a guest in a real CPython sub-interpreter cell. + + Fails closed on builds without ``concurrent.interpreters`` rather than + degrading to the thread backend: the two have different isolation, and + silently substituting one for the other is what the backend rename was + meant to stop. + """ + subinterpreter.require_available() + spec = subinterpreter.CellSpec.build(allowed_imports) + cell_sandbox = subinterpreter.SubinterpreterSandbox( + name, + pool=self._cell_pool(), + spec=spec, + wall_time_ms=wall_time_ms, + ) + with self._lock: + existing = self._cell_sandboxes.get(name) + if existing is not None and existing.is_alive(): + cell_sandbox.close() + raise RuntimeError(f"sandbox '{name}' already exists") + self._cell_sandboxes[name] = cell_sandbox + return Sandbox(cell_sandbox, self) + + def _cell_pool(self) -> "subinterpreter.CellPool": + with self._lock: + if self._pool is None: + self._pool = subinterpreter.CellPool() + return self._pool + def _spawn_microvm(self, name: str) -> Sandbox: """Admit a microVM sandbox, failing closed until the launcher lands. @@ -737,10 +796,17 @@ def shutdown(self, cap: RootCapability = ROOT) -> None: warm = list(self._warm_pool) self._warm_pool.clear() procs = list(self._process_sandboxes.values()) + cells = list(self._cell_sandboxes.values()) + self._cell_sandboxes.clear() + pool, self._pool = self._pool, None for sb in sandboxes + warm: sb.stop() for proc in procs: proc.stop() + for cell in cells: + cell.stop() + if pool is not None: + pool.close() self._cleanup() def quarantine(self, name: str, reason: str) -> None: @@ -815,6 +881,11 @@ def _cleanup(self) -> None: proc.reap() self._release_tenant_reservation(proc) del self._process_sandboxes[n] + dead_cells = [ + n for n, c in self._cell_sandboxes.items() if not c.is_alive() + ] + for n in dead_cells: + del self._cell_sandboxes[n] _supervisor: Supervisor | None = None diff --git a/tests/test_subinterpreter_backend.py b/tests/test_subinterpreter_backend.py new file mode 100644 index 0000000..174332d --- /dev/null +++ b/tests/test_subinterpreter_backend.py @@ -0,0 +1,520 @@ +"""Tests for the real sub-interpreter cell backend. + +Two groups. The first needs CPython 3.14+ and exercises actual interpreters; +the second runs everywhere and covers the parts that decide behaviour without +one -- the fail-closed path, the spec, and the pool's bookkeeping -- so the +3.11-3.13 matrix still gets coverage of the logic it can reach. +""" + +import subprocess +import sys +import textwrap +from pathlib import Path + +import pytest + +from pyisolate import errors +from pyisolate.runtime import subinterpreter as S + +ROOT = Path(__file__).resolve().parents[1] + +requires_interpreters = pytest.mark.skipif( + not S.is_available(), + reason=f"needs CPython {S.MIN_PYTHON[0]}.{S.MIN_PYTHON[1]}+ for concurrent.interpreters", +) + + +# --- availability --------------------------------------------------------- + + +def test_min_python_is_the_public_api_not_the_private_one(): + """3.12/3.13's _interpreters heap-corrupts on realistic import surfaces.""" + assert S.MIN_PYTHON == (3, 14) + + +def test_is_available_tracks_the_running_build(): + expected = sys.version_info[:2] >= S.MIN_PYTHON + assert S.is_available() is expected + + +@pytest.mark.skipif(S.is_available(), reason="this build has sub-interpreters") +def test_fails_closed_rather_than_degrading_to_a_thread(): + """Substituting a different isolation model silently is the whole problem.""" + with pytest.raises(errors.SandboxError) as excinfo: + S.require_available() + message = str(excinfo.value) + assert "3.14" in message + assert "backend='thread'" in message + assert "backend='process'" in message + + +# --- CellSpec (no interpreter needed) ------------------------------------- + + +def test_spec_preimports_everything_allowed_by_default(): + spec = S.CellSpec.build(["json", "math"]) + assert spec.allowed_imports == frozenset({"json", "math"}) + assert spec.preimport == ("json", "math") + + +def test_spec_rejects_preimporting_a_module_policy_forbids(): + with pytest.raises(ValueError, match="not in allowed_imports"): + S.CellSpec(allowed_imports=frozenset({"math"}), preimport=("os",)) + + +def test_spec_allows_parents_of_allowed_submodules(): + """``import a.b`` imports ``a`` first, so the parent has to resolve.""" + spec = S.CellSpec.build(["email.message"]) + assert spec.allows("email.message") + assert spec.allows("email") + assert not spec.allows("os") + assert not spec.allows("emailx") + + +def test_spec_is_hashable_so_the_pool_can_key_on_it(): + a = S.CellSpec.build(["json"]) + b = S.CellSpec.build(["json"]) + c = S.CellSpec.build(["json", "math"]) + assert a == b and hash(a) == hash(b) + assert len({a, b, c}) == 2 + + +def test_runtime_imports_are_allowed_so_the_queue_can_work(): + """Blocking these breaks CPython's own machinery, not the guest.""" + assert "pickle" in S._RUNTIME_IMPORTS + assert "traceback" in S._RUNTIME_IMPORTS + + +# --- pool bookkeeping ----------------------------------------------------- + + +def test_pool_rejects_nonsense_sizes(): + with pytest.raises(ValueError): + S.CellPool(warm_per_spec=-1) + with pytest.raises(ValueError): + S.CellPool(max_warm=-1) + + +@requires_interpreters +def test_pool_reuses_a_warm_cell_instead_of_creating_one(): + with S.CellPool(warm_per_spec=0) as pool: + spec = S.CellSpec.build(["math"]) + assert pool.prewarm(spec, 2) == 2 + assert pool.stats()["warm"] == 2 + + first = pool.acquire(spec) + assert pool.stats()["reused"] == 1 + assert pool.stats()["warm"] == 1 + first.retire() + + +@requires_interpreters +def test_pool_creates_on_a_miss(): + with S.CellPool(warm_per_spec=0) as pool: + cell = pool.acquire(S.CellSpec.build(["math"])) + stats = pool.stats() + assert stats["created"] == 1 + assert stats["reused"] == 0 + cell.retire() + + +@requires_interpreters +def test_a_warm_cell_is_not_handed_to_a_different_spec(): + """A cell is only interchangeable with one warmed the same way.""" + with S.CellPool(warm_per_spec=0) as pool: + warmed = S.CellSpec.build(["math"]) + other = S.CellSpec.build(["json"]) + pool.prewarm(warmed, 1) + cell = pool.acquire(other) + assert pool.stats()["reused"] == 0 + assert pool.stats()["created"] == 2 + cell.retire() + + +@requires_interpreters +def test_release_retires_rather_than_returning_to_the_pool(): + """There is no reset, so a used cell cannot be given to another tenant.""" + with S.CellPool(warm_per_spec=0) as pool: + spec = S.CellSpec.build(["math"]) + cell = pool.acquire(spec) + pool.release(cell) + assert cell.state is S.CellState.RETIRED + assert pool.stats()["retired"] == 1 + assert pool.stats()["warm"] == 0 + + +@requires_interpreters +def test_acquire_after_close_is_refused(): + pool = S.CellPool(warm_per_spec=0) + pool.close() + with pytest.raises(errors.SandboxError, match="closed"): + pool.acquire(S.CellSpec.build([])) + + +# --- the isolation the backend exists for --------------------------------- + + +@requires_interpreters +def test_cells_do_not_share_globals(): + spec = S.CellSpec.build(["math"]) + a, b = S.Cell(spec), S.Cell(spec) + try: + a.exec("leaked = 'from A'") + with pytest.raises(Exception) as excinfo: + b.exec("post(leaked)") + assert "leaked" in str(excinfo.value) + finally: + a.retire() + b.retire() + + +@requires_interpreters +def test_cells_do_not_share_sys_modules(): + """The actual difference from the thread backend.""" + spec = S.CellSpec.build(["json", "math"]) + a, b = S.Cell(spec), S.Cell(spec) + try: + a.exec("import json; json.MARKER = 'set by A'") + b.exec("import json; post(hasattr(json, 'MARKER'))") + assert b.drain() == [("post", False)] + finally: + a.retire() + b.retire() + + +@requires_interpreters +def test_the_import_hook_does_not_touch_the_host_interpreter(): + import builtins + + before = builtins.__import__ + cell = S.Cell(S.CellSpec.build(["math"])) + try: + cell.exec("import math") + assert builtins.__import__ is before + # ...and the host can still import something the cell could not. + import os # noqa: F401 + finally: + cell.retire() + + +@requires_interpreters +def test_denied_import_raises_inside_the_cell(): + cell = S.Cell(S.CellSpec.build(["math"])) + try: + with pytest.raises(Exception, match="not permitted by policy"): + cell.exec("import os") + finally: + cell.retire() + + +@requires_interpreters +def test_allowed_import_works(): + cell = S.Cell(S.CellSpec.build(["math"])) + try: + cell.exec("from math import sqrt; post(sqrt(9))") + assert cell.drain() == [("post", 3.0)] + finally: + cell.retire() + + +# --- the message contract ------------------------------------------------- + + +@requires_interpreters +def test_messages_cross_as_json_so_the_supervisor_never_unpickles(): + """The queue pickles anything not natively shareable; a str is shareable.""" + cell = S.Cell(S.CellSpec.build([])) + try: + cell.exec("post({'a': [1, 2], 'b': None})") + raw = cell._outbox.get_nowait() + assert isinstance(raw, str) + assert raw == '["post", {"a": [1, 2], "b": null}]' + finally: + cell.retire() + + +@requires_interpreters +def test_posting_something_unserialisable_fails_at_the_call(): + """A clear TypeError beats an opaque NotShareableError from the runtime.""" + cell = S.Cell(S.CellSpec.build([])) + try: + with pytest.raises(Exception) as excinfo: + cell.exec("post(object())") + assert "JSON serializable" in str(excinfo.value) + finally: + cell.retire() + + +@requires_interpreters +def test_log_metric_and_request_all_reach_the_supervisor(): + cell = S.Cell(S.CellSpec.build([])) + try: + cell.exec( + "log('info', 'hello', k=1)\n" + "metric('m', 3)\n" + "request('read_path', '/tmp/x', mode='r')\n" + ) + kinds = dict(cell.drain()) + assert kinds["log"] == {"level": "info", "message": "hello", "fields": {"k": 1}} + assert kinds["metric"] == {"name": "m", "value": 3} + assert kinds["request"] == { + "capability": "read_path", + "args": ["/tmp/x"], + "kwargs": {"mode": "r"}, + } + finally: + cell.retire() + + +def test_malformed_cell_messages_are_dropped_not_propagated(): + """Guest code can reach the outbox, so the frame is validated not trusted.""" + assert S.Cell._decode("not json") is None + assert S.Cell._decode(b"bytes") is None + assert S.Cell._decode('{"not": "a list"}') is None + assert S.Cell._decode('["one element"]') is None + assert S.Cell._decode("[1, 2]") is None + assert S.Cell._decode('["post", 42]') == ("post", 42) + + +# --- reclaim --------------------------------------------------------------- + + +@requires_interpreters +def test_a_retired_cell_cannot_be_reused(): + cell = S.Cell(S.CellSpec.build([])) + assert cell.retire() is True + with pytest.raises(errors.SandboxError, match="retired"): + cell.exec("x = 1") + + +@requires_interpreters +def test_an_abandoned_cell_cannot_be_reused(): + cell = S.Cell(S.CellSpec.build([])) + try: + cell.abandon("test") + assert cell.state is S.CellState.ABANDONED + assert cell.is_usable() is False + with pytest.raises(errors.SandboxError, match="abandoned"): + cell.exec("x = 1") + finally: + cell.retire() + + +#: Scenarios below run in a *fresh* interpreter, not a fork. +#: +#: They strand a thread inside a spinning interpreter, which cannot be +#: reclaimed -- that is the thing being tested -- so running them in-process +#: would hang pytest at exit. ``os.fork()`` is not the answer either: forking a +#: process that already hosts sub-interpreters and their threads segfaults, as +#: it does when these tests follow others in the same session. A fresh process +#: is the only clean kill domain, which is the same conclusion a deployment +#: reaches: pre-fork the workers, then create cells inside them. +_CHILD_PREAMBLE = """ +import sys, time +sys.path.insert(0, %r) +from pyisolate import errors +from pyisolate.runtime import subinterpreter as S +""" + + +def _run_in_fresh_process( + body_source: str, timeout: float = 60.0 +) -> subprocess.CompletedProcess: + script = (_CHILD_PREAMBLE % str(ROOT)) + textwrap.dedent(body_source) + return subprocess.run( + [sys.executable, "-W", "ignore", "-c", script], + capture_output=True, + text=True, + timeout=timeout, + ) + + +@requires_interpreters +def test_timeout_abandons_the_cell_and_says_so(): + """A running interpreter cannot be reclaimed, so do not claim it was. + + The spun-up thread stays pinned for the life of the process. That is the + honest outcome, and the reason a deployment that must survive runaway + guests needs a process-level kill domain. + """ + result = _run_in_fresh_process( + r""" + pool = S.CellPool(warm_per_spec=0) + sandbox = S.SubinterpreterSandbox( + "runaway", pool=pool, spec=S.CellSpec.build([]), wall_time_ms=250 + ) + try: + sandbox.exec("while True:\n pass") + except errors.WallTimeExceeded as exc: + assert "abandoned" in str(exc), exc + else: + raise AssertionError("runaway exec returned") + assert pool.stats()["abandoned"] == 1, pool.stats() + assert sandbox.is_alive() is False + # The pool stays usable for everyone else; only the one cell is lost. + other = pool.acquire(S.CellSpec.build([])) + other.exec("post(1)") + assert other.drain() == [("post", 1)] + other.retire() + print("OK") + # The stranded thread would otherwise keep this process alive. + import os + os._exit(0) + """ + ) + assert "OK" in result.stdout, result.stderr + assert result.returncode == 0, result.stderr + + +@requires_interpreters +def test_kill_reports_false_while_running_rather_than_lying(): + result = _run_in_fresh_process( + r""" + pool = S.CellPool(warm_per_spec=0) + cell = pool.acquire(S.CellSpec.build([])) + cell.exec_in_thread("while True:\n pass") + # Wait for the runtime to report the interpreter as executing, not + # just for our own flag: exec_in_thread sets RUNNING before its thread + # has entered the interpreter. + deadline = time.monotonic() + 5.0 + while not cell._is_running() and time.monotonic() < deadline: + time.sleep(0.005) + assert cell._is_running(), cell.state + assert cell.kill() is False + assert cell.retire() is False + assert cell.state is S.CellState.ABANDONED, cell.state + print("OK") + import os + os._exit(0) + """ + ) + assert "OK" in result.stdout, result.stderr + assert result.returncode == 0, result.stderr + + +# --- the sandbox handle --------------------------------------------------- + + +@requires_interpreters +def test_sandbox_round_trip(): + with S.CellPool(warm_per_spec=0) as pool: + sandbox = S.SubinterpreterSandbox( + "demo", pool=pool, spec=S.CellSpec.build(["math"]) + ) + sandbox.exec("from math import sqrt; post(sqrt(2))") + assert sandbox.recv() == pytest.approx(1.4142135623730951) + assert sandbox.is_alive() is True + sandbox.close() + assert sandbox.is_alive() is False + + +@requires_interpreters +def test_sandbox_surfaces_broker_requests_for_the_supervisor(): + with S.CellPool(warm_per_spec=0) as pool: + sandbox = S.SubinterpreterSandbox("demo", pool=pool, spec=S.CellSpec.build([])) + sandbox.exec("request('read_path', '/etc/hosts')") + requests = sandbox.get_broker_requests() + assert requests == [ + {"capability": "read_path", "args": ["/etc/hosts"], "kwargs": {}} + ] + sandbox.close() + + +@requires_interpreters +def test_sandbox_stats_expose_the_cell_and_the_pool(): + with S.CellPool(warm_per_spec=0) as pool: + sandbox = S.SubinterpreterSandbox("demo", pool=pool, spec=S.CellSpec.build([])) + stats = sandbox.stats() + assert stats["backend"] == "subinterpreter" + assert isinstance(stats["cell_id"], int) + assert stats["cell_state"] == "idle" + assert stats["pool"]["created"] == 1 + sandbox.close() + + +@requires_interpreters +def test_recv_times_out_rather_than_blocking_forever(): + with S.CellPool(warm_per_spec=0) as pool: + sandbox = S.SubinterpreterSandbox("demo", pool=pool, spec=S.CellSpec.build([])) + with pytest.raises(errors.TimeoutError): + sandbox.recv(timeout=0.05) + sandbox.close() + + +# --- the surface the Sandbox handle delegates to -------------------------- + + +@requires_interpreters +def test_call_takes_a_dotted_name_and_returns_the_result(): + with S.CellPool(warm_per_spec=0) as pool: + sandbox = S.SubinterpreterSandbox( + "demo", pool=pool, spec=S.CellSpec.build(["math"]) + ) + assert sandbox.call("math.factorial", 5) == 120 + sandbox.close() + + +@requires_interpreters +def test_call_respects_the_import_allow_list(): + """Resolution goes through the cell's guarded __import__, not the host's.""" + with S.CellPool(warm_per_spec=0) as pool: + sandbox = S.SubinterpreterSandbox( + "demo", pool=pool, spec=S.CellSpec.build(["math"]) + ) + with pytest.raises(Exception, match="not permitted by policy"): + sandbox.call("os.getpid") + sandbox.close() + + +@requires_interpreters +def test_call_rejects_a_bare_name(): + with S.CellPool(warm_per_spec=0) as pool: + sandbox = S.SubinterpreterSandbox("demo", pool=pool, spec=S.CellSpec.build([])) + with pytest.raises(ValueError, match="dotted name"): + sandbox.call("factorial") + with pytest.raises(TypeError, match="dotted name"): + sandbox.call(None) # type: ignore[arg-type] + sandbox.close() + + +@requires_interpreters +def test_reset_is_refused_with_the_reason(): + """Not a stub: a cell genuinely cannot be returned to a pristine state.""" + with S.CellPool(warm_per_spec=0) as pool: + sandbox = S.SubinterpreterSandbox("demo", pool=pool, spec=S.CellSpec.build([])) + with pytest.raises(NotImplementedError, match="cannot be reset"): + sandbox.reset() + sandbox.close() + + +@requires_interpreters +def test_snapshot_carries_configuration_not_guest_state(): + with S.CellPool(warm_per_spec=0) as pool: + sandbox = S.SubinterpreterSandbox( + "demo", pool=pool, spec=S.CellSpec.build(["math"]), wall_time_ms=1000 + ) + assert sandbox.snapshot() == { + "name": "demo", + "backend": "subinterpreter", + "allowed_imports": ["math"], + "wall_time_ms": 1000, + } + sandbox.close() + + +@requires_interpreters +def test_quarantine_abandons_the_cell(): + with S.CellPool(warm_per_spec=0) as pool: + sandbox = S.SubinterpreterSandbox("demo", pool=pool, spec=S.CellSpec.build([])) + sandbox.quarantine("policy breach") + assert pool.stats()["abandoned"] == 1 + assert sandbox.is_alive() is False + + +@requires_interpreters +def test_tracing_is_refused_rather_than_silently_empty(): + with S.CellPool(warm_per_spec=0) as pool: + sandbox = S.SubinterpreterSandbox("demo", pool=pool, spec=S.CellSpec.build([])) + with pytest.raises(NotImplementedError, match="process-backend"): + sandbox.enable_tracing() + sandbox.close() diff --git a/tests/test_supervisor.py b/tests/test_supervisor.py index 47ca6fd..5fea695 100644 --- a/tests/test_supervisor.py +++ b/tests/test_supervisor.py @@ -10,6 +10,7 @@ import pyisolate as iso from pyisolate.bpf.manager import BPFManager +from pyisolate.runtime import subinterpreter def test_module_import_is_lazy(monkeypatch): @@ -187,8 +188,13 @@ def test_spawn_backend_is_explicit_thread(): sb = iso.spawn("backend-thread", backend="thread") try: assert sb.backend == "thread" - assert iso.SUPPORTED_BACKENDS == ("thread", "process", "microvm") - assert iso.IMPLEMENTED_BACKENDS == ("thread", "process") + assert iso.SUPPORTED_BACKENDS == ( + "thread", + "subinterpreter", + "process", + "microvm", + ) + assert iso.IMPLEMENTED_BACKENDS == ("thread", "subinterpreter", "process") assert iso.DEFAULT_BACKEND == "thread" finally: sb.close() @@ -205,21 +211,35 @@ def test_default_backend_does_not_warn(): sb.close() -def test_subinterpreter_backend_is_a_deprecated_alias_for_thread(): - """The old spelling keeps working, warns, and selects the same runtime.""" - with pytest.warns(DeprecationWarning, match="thread"): - sb = iso.spawn("backend-alias", backend="subinterpreter") +def test_subinterpreter_is_now_a_backend_in_its_own_right(): + """The alias is gone; the name selects the real sub-interpreter runtime.""" + assert iso.DEPRECATED_BACKEND_ALIASES == {} + assert "subinterpreter" in iso.SUPPORTED_BACKENDS + assert "subinterpreter" in iso.IMPLEMENTED_BACKENDS + + +@pytest.mark.skipif( + not subinterpreter.is_available(), + reason="needs CPython 3.14+ for concurrent.interpreters", +) +def test_subinterpreter_backend_runs_a_real_cell(): + sb = iso.spawn("backend-cell", backend="subinterpreter", allowed_imports=["math"]) try: - assert sb.backend == "thread" + assert sb.backend == "subinterpreter" + sb.exec("from math import sqrt; post(sqrt(4))") + assert sb.recv(timeout=5) == 2.0 finally: sb.close() - assert iso.DEPRECATED_BACKEND_ALIASES == {"subinterpreter": "thread"} -def test_deprecated_alias_is_not_advertised_as_supported(): - """It resolves, but it is not one of the names the API offers.""" - assert "subinterpreter" not in iso.SUPPORTED_BACKENDS - assert "subinterpreter" not in iso.IMPLEMENTED_BACKENDS +@pytest.mark.skipif( + subinterpreter.is_available(), + reason="this build has sub-interpreters", +) +def test_subinterpreter_backend_fails_closed_below_314(): + """It must not quietly hand back a thread, which isolates differently.""" + with pytest.raises(iso.SandboxError, match="3.14"): + iso.spawn("backend-cell", backend="subinterpreter") def test_unknown_backend_still_rejects_without_warning(): From 9ec9622200ba068a7c7908fc23401152682482de Mon Sep 17 00:00:00 2001 From: Claude Date: Wed, 16 Sep 2026 23:29:57 +0000 Subject: [PATCH 2/2] Cover CPython 3.14 and the sub-interpreter backend in CI The sub-interpreter backend needs 3.14+, and nothing in the matrix ran it: unit tests stopped at 3.13 and the only free-threaded job was 3.13t, which cannot create a cell at all. Add 3.14 to the unit matrix, and a `sub-interpreter cells / py3.14t` job that runs the backend suite on a free-threaded build. That job asserts up front that the interpreter really is a free-threaded 3.14 before running any tests. Every sub-interpreter test skips itself when the build cannot run it -- which is correct for the 3.11-3.13 matrix, but it means a runner that silently resolved to an older Python would skip everything and report the job green having tested nothing. Failing loudly is the difference between a job that covers the backend and a job that looks like it does. Dependency installation follows the existing 3.13t job: install without the runtime deps that may not have free-threaded wheels, then re-add the pure-Python ones the import path needs. Nothing in this job's scope needs real crypto, and the suite's stub covers it. Co-Authored-By: Claude Opus 5 Claude-Session: https://claude.ai/code/session_01Fs1hTtmF4Hm9h617AG9Gse --- .github/workflows/ci.yml | 36 +++++++++++++++++++++++++++++++++++- CHANGELOG.md | 7 +++++++ pyproject.toml | 1 + 3 files changed, 43 insertions(+), 1 deletion(-) diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml index b58adeb..682cedb 100644 --- a/.github/workflows/ci.yml +++ b/.github/workflows/ci.yml @@ -34,7 +34,7 @@ jobs: fail-fast: false matrix: os: [ubuntu-22.04, ubuntu-24.04] - python-version: ["3.11", "3.12", "3.13"] + python-version: ["3.11", "3.12", "3.13", "3.14"] steps: - uses: actions/checkout@v4 - uses: actions/setup-python@v5 @@ -101,6 +101,40 @@ jobs: continue-on-error: true run: pytest -q tests/test_matrix_hardening.py -m "race and no_gil" + tests-subinterpreter: + name: sub-interpreter cells / py3.14t + runs-on: ubuntu-24.04 + steps: + - uses: actions/checkout@v4 + - uses: actions/setup-python@v5 + with: + python-version: "3.14t" + - name: Install dependencies + # Same shape as the 3.13t job: install without the runtime deps that + # may not have free-threaded wheels yet, then re-add the pure-Python + # ones the import path needs. The suite's crypto stub (tests/conftest.py) + # covers the rest, and nothing in this job's scope needs real crypto. + run: | + python -m pip install -e . --no-deps + python -m pip install pytest pyyaml platformdirs + - name: Verify this build actually has sub-interpreters + # Every sub-interpreter test skips itself when the build cannot run it. + # That is right for the 3.11-3.13 matrix, but it means this job would + # report green on a runner that silently resolved to an older Python + # while testing nothing. Fail loudly instead. + run: | + python - <<'PY' + import sys + from pyisolate.runtime import subinterpreter as s + assert s.is_available(), f"no concurrent.interpreters on {sys.version}" + assert not sys._is_gil_enabled(), "expected a free-threaded build" + print("ok:", sys.version) + PY + - name: Run the sub-interpreter backend suite + run: pytest -q tests/test_subinterpreter_backend.py tests/test_supervisor.py + - name: Validate the no-GIL readiness axis on 3.14t + run: pytest -q tests/test_nogil.py + tests-soak: name: soak / 2k spawn-kill cycles if: github.event_name == 'schedule' diff --git a/CHANGELOG.md b/CHANGELOG.md index 6b5c73a..8ab2d57 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -31,6 +31,13 @@ guarantees; **no release should be treated as a hardened security boundary**. - `pyisolate[operator]` optional-dependency group for the Kubernetes operator. ### Changed +- CI covers CPython 3.14: the unit matrix gains `3.14`, and a new + `sub-interpreter cells / py3.14t` job runs the sub-interpreter backend on a + free-threaded build. That job asserts the interpreter really is a + free-threaded 3.14 before running anything, because every sub-interpreter + test skips itself when the build cannot run it -- correct for the 3.11-3.13 + matrix, but it would otherwise let the job report green having tested + nothing. - `backend="subinterpreter"` is renamed to `backend="thread"`, which is what it has always run, and the `subinterpreter` name now selects the real sub-interpreter backend. `DEPRECATED_BACKEND_ALIASES` is exported alongside diff --git a/pyproject.toml b/pyproject.toml index 6a99980..17500ad 100644 --- a/pyproject.toml +++ b/pyproject.toml @@ -25,6 +25,7 @@ classifiers = [ "Programming Language :: Python :: 3.11", "Programming Language :: Python :: 3.12", "Programming Language :: Python :: 3.13", + "Programming Language :: Python :: 3.14", "Topic :: Security", "Topic :: Software Development :: Libraries :: Python Modules", "Topic :: System :: Systems Administration",