Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -28,7 +28,7 @@ golden fixtures), the durable runtime (crash-recoverable, idempotent), isolated
model routing, disposable projections, the local API + trusted Shell, the three
daily-driver workflows delivered **in the Shell** (grounded Q&A, model-planned durable
agents, isolated coding workspace), and ops (backup/restore/doctor). Current main:
**1242 passed / 25 skipped** plus **14 Playwright specs across 10 files** driving the real
**1248 passed / 25 skipped** plus **14 Playwright specs across 10 files** driving the real
rendered Shell; the kernel import boundary and the crash-recovery / revocation /
backup-restore / approval-gating end-to-end scenarios pass headless. See
[`docs/architecture/system-overview.md`](docs/architecture/system-overview.md), the
Expand Down
71 changes: 42 additions & 29 deletions SECURITY.md
Original file line number Diff line number Diff line change
Expand Up @@ -308,41 +308,50 @@ weighing this jail needs them more than they need the three above.
would be absent at once; `containment_report` emits a loud top-level `warnings` entry for
exactly that combination. Scope, measured footprint, and the phased plan:
`docs/design/syscall-filtering.md`.
- **The chroot is escapable, on every architecture and every profile.** `chroot()` does not
move the caller's cwd, and `unshare(CLONE_NEWUSER)` grants the worker `CAP_SYS_CHROOT`
over its own namespace — so a hostile effect can `chroot()` into a subdirectory, walk `..`
out of it, and re-root at the real filesystem root. Verified against a real `PURE` worker:
it reads `/etc/passwd`, lists the host home including `~/.ssh`, and **writes host files**.
The PID and network namespaces still hold (a host PID is `ESRCH`; there is no route out),
so this is a filesystem escape — but the filesystem is where the Weft database, the
keystore, and `~/.ssh` live. Blast radius follows the `uid_map`: the worker acts on host
files as the parent's real uid, so under an unprivileged service account it still holds
everything the Decima user holds. **There is no configuration in which this is contained.**
`test_worker_cannot_read_dot_ssh`, `test_worker_cannot_read_etc_passwd_by_absolute_path`,
and `test_weft_db_access_attempt_from_worker_fails` assert a real refusal on a real path
and pass today only because no test attempts a second `chroot`. Analysis and the fix
options (`pivot_root`, or dropping `CAP_SYS_CHROOT` once the jail is built):
`docs/design/syscall-filtering.md` §4.3.
- **A `WORKSPACE` worker can write a read-only workspace and reach above the containment
root.** The `O_PATH` fd the parent pins on the declared subtree is passed as `argv[4]`,
stays open for the child's whole life, and is visible to the untrusted implementation via
`sys.argv`. It cannot be `fchdir`'d, but it is a valid `dirfd` for `openat` — and it
references the *original* mount, where `MS_RDONLY`/`nosuid`/`nodev`/`noexec` were never
set. Verified against a read-only mount whose posture read back correctly: a write through
`/workspace` is refused, a write through the fd **succeeds**, and
`openat(fd, "../..")` lists the host root. So the read-only tier and `mount.py`'s
containment-root rule are both defeated by an inherited descriptor rather than by any
weakness in the mount itself. The fix is to close the fd after `bind_workspace` and before
the implementation is exec'd; no syscall filter can close this one, since a filter cannot
tell where a `dirfd` points. `docs/design/syscall-filtering.md` §4.4.
- **CLOSED (wave S0) — the chroot was escapable, on every architecture and every profile.**
`chroot()` does not move the caller's cwd, and `unshare(CLONE_NEWUSER)` grants the worker
`CAP_SYS_CHROOT` over its own namespace, so a hostile effect could `chroot()` into a
subdirectory, walk `..` out of it, and re-root at the real filesystem root. Verified against
a real `PURE` worker: it read `/etc/passwd`, listed the host home including `~/.ssh`, and
**wrote host files** — as the parent's real uid, so no service account contained it.
The jail now uses **`pivot_root` + `umount2(MNT_DETACH)`** instead: the host tree is not
present in the worker's mount namespace at all afterwards. The fix is deliberately
STRUCTURAL rather than a prohibition — a hostile effect may still call `chroot` as often as
it likes, and every path it can name resolves inside the jail, so the property survives a
regained capability and does not depend on the syscall filter (absent on x86_64) denying
anything. An architecture with no known `pivot_root` syscall number REFUSES rather than
falling back to `chroot`. `tests/adversarial/test_jail_escape.py` runs the full hostile
sequence and asserts it lands in the jail; all five of its escape-A tests go red when the
fix is reverted. Analysis: `docs/design/syscall-filtering.md` §4.3.
- **CLOSED (wave S0) — a `WORKSPACE` worker could write a read-only workspace and reach above
the containment root.** The `O_PATH` fd the parent pins on the declared subtree was passed
as `argv[4]`, stayed open for the child's whole life, and was visible to the untrusted
implementation via `sys.argv`. It cannot be `fchdir`'d, but it is a valid `dirfd` for
`openat` — and it referenced the *original* mount, where `MS_RDONLY`/`nosuid`/`nodev`/
`noexec` were never set, so the read-only tier and the containment-root rule were both
defeated by an inherited descriptor rather than by any weakness in the mount. No syscall
filter could have fixed it: seccomp cannot tell where a `dirfd` points. The fd is now closed
after `bind_workspace` and **before any untrusted byte runs**, the closure is read back with
`fstat` (a descriptor believed closed but still open fails the worker closed), and
`allowed_fds` no longer lists it.
- A `WORKSPACE` dispatch handed **no** subtree is **refused** (`IsolationError`), never
downgraded to the write-less `PURE` jail under a profile name that promises otherwise.
- The **grant does not choose the blast radius.** For a `workspace_write` organ the root is
an operator-supplied deployment fact (`executor.generated_code_effect(workspace_root=…)`,
default `None` = concede nothing); a capability caveat may only name a subtree *beneath*
it, and an absolute path, a `..` component, or a symlink leaving the root are refused.
- Having an executor is **not** being auto-promotable. `promotion.SIGNER_POLICY` still
requires a human attestation for `workspace_write`, and `anchors.SIGNABLE_TIERS` still
excludes it, so the Reckoner cannot promote one on its own.

**What changed, and what an operator may now rely on.** The workspace bind-mount
(`workspace_bind_mount`) used to be on this list. It is now REAL and enforced for the
`WORKSPACE` profile: one caller-declared host subtree is `MS_BIND`-mounted at `/workspace`
inside the worker's mount namespace before the `chroot`, always `nosuid`+`nodev`+`noexec`
inside the worker's mount namespace before the root pivot, always `nosuid`+`nodev`+`noexec`
(plus `MS_RDONLY` for a read-only mount), with the mounted inode re-verified against an
`O_PATH` fd the parent pinned — so the source cannot be swapped between the containment
check and the mount — and the posture read back from `statvfs`. Three things bound it:
check and the mount — the posture read back from `statvfs`, and that pin fd **closed before
any untrusted byte runs** (wave S0). Three things bound it:

- A `WORKSPACE` dispatch handed **no** subtree is **refused** (`IsolationError`), never
downgraded to the write-less `PURE` jail under a profile name that promises otherwise.
Expand All @@ -354,6 +363,10 @@ check and the mount — and the posture read back from `statvfs`. Three things b
requires a human attestation for `workspace_write`, and `anchors.SIGNABLE_TIERS` still
excludes it, so the Reckoner cannot promote one on its own.

And the `workspace_write` TIER itself is still not mapped in `executor.TIER_PROFILES`. It was
withheld because the jail was escapable; S0 closed that, so what remains is a deliberate
sequencing decision rather than a defect — see `executor.py`'s table comment.

## Automated guardrails

- `tests/architecture/test_import_boundaries.py` fails the build if the trusted
Expand Down
118 changes: 116 additions & 2 deletions decima/workers/execution.py
Original file line number Diff line number Diff line change
Expand Up @@ -773,9 +773,89 @@ def apply_namespaces():
if cfg.get("workspace") is not None and not ws.get("engaged"):
report["detail"] = "workspace bind refused: %s" % ws.get("detail")
return report
if libc.chroot(scratch.encode()) != 0:
report["detail"] = "chroot failed (errno %d)" % ctypes.get_errno()
# -- PIVOT_ROOT, NOT CHROOT (wave S0) -------------------------------------------
# `chroot` was an escapable jail and this is the whole reason S0 exists. chroot()
# does not move the caller's cwd, and a task that is root in its own user namespace
# holds CAP_SYS_CHROOT unconditionally — so `mkdir e; chroot e; chdir ../../..;
# chroot .` re-rooted on the HOST filesystem. Verified from a PURE worker before this
# change: read /etc/passwd, listed host /, wrote a host file, effect SUCCEEDED.
#
# The fix is not to forbid the second chroot; it is to make it POINT NOWHERE.
# pivot_root moves the process's root to the scratch mount and puts the old root at a
# known path, which we then detach: after `umount2(MNT_DETACH)` the host tree is no
# longer reachable from this mount namespace AT ALL. A hostile effect may still call
# chroot as often as it likes — every path it can name resolves inside the jail,
# because the host mounts are gone rather than merely stepped over. That is a
# structural containment property rather than a capability check, which is what makes
# it worth the extra syscalls: it survives a regained capability and it does not
# depend on the seccomp filter (absent on x86_64) denying anything.
#
# Three preconditions, all already satisfied here, which is why the call sits exactly
# at this point: we are in our own mount namespace (CLONE_NEWNS above), the tree is
# MS_REC|MS_PRIVATE (so nothing propagates to the host mount table), and the workspace
# bind has already happened while the host-side source path was still resolvable.
# pivot_root additionally requires new_root to BE a mount point and to differ from the
# current root's mount, hence the bind of scratch onto itself first.
MS_BIND = 0x1000
MNT_DETACH = 2
# No glibc wrapper for pivot_root on every libc we support, so it goes through
# syscall(2). The number is per-arch: an ARCH TABLE, never a bare constant — a wrong
# number is a silently different syscall.
_PIVOT_ROOT_NR = {"x86_64": 155, "aarch64": 41, "armv7l": 218, "s390x": 217}
pivot_nr = _PIVOT_ROOT_NR.get(os.uname().machine)
if pivot_nr is None:
report["detail"] = (
"no pivot_root syscall number known for arch %r, and chroot alone is an "
"escapable jail (S0) — refusing rather than running with weaker containment "
"than the profile promises" % os.uname().machine
)
return report
try:
scratch_ino = os.stat(scratch).st_ino
except OSError as e:
report["detail"] = "scratch stat failed before pivot: %s" % e
return report
if libc.mount(scratch.encode(), scratch.encode(), None, MS_BIND | MS_REC, None) != 0:
report["detail"] = "bind of scratch onto itself failed (errno %d)" % (
ctypes.get_errno()
)
return report
old_root = os.path.join(scratch, ".decima-oldroot")
try:
os.mkdir(old_root, 0o700)
except FileExistsError:
pass
except OSError as e:
report["detail"] = "oldroot mkdir failed: %s" % e
return report
if libc.syscall(pivot_nr, scratch.encode(), old_root.encode()) != 0:
report["detail"] = "pivot_root failed (errno %d)" % ctypes.get_errno()
return report
os.chdir("/")
# DETACH, then verify. An undetached old root would leave the whole host tree one
# `chdir` away, so this is not cleanup — it is the containment step, and a failure
# here fails the worker closed rather than logging a warning.
if libc.umount2(b"/.decima-oldroot", MNT_DETACH) != 0:
report["detail"] = "old-root detach failed (errno %d)" % ctypes.get_errno()
return report
try:
os.rmdir("/.decima-oldroot")
except OSError:
# Cosmetic only: the mount is already detached, so an empty directory is all
# that can remain. Never fatal — it would fail a contained worker for tidiness.
pass
# Read the property back rather than trusting two return codes: `/` must now BE the
# scratch directory, compared against the inode captured before the pivot. Note what
# this is not — comparing `stat("/")` with `stat(".")` would be vacuous, because the
# chdir above makes them the same by construction whether or not the pivot took.
try:
if os.stat("/").st_ino != scratch_ino:
report["detail"] = "post-pivot root is not the scratch jail (read-back)"
return report
except OSError as e:
report["detail"] = "post-pivot root read-back failed: %s" % e
return report
report["pivoted"] = True
# A bound worker's cwd IS its workspace: that is what makes the bind load-bearing
# rather than decorative — ordinary relative-path writes land on the host subtree.
os.chdir(ws["jail_path"] if ws.get("engaged") else "/")
Expand Down Expand Up @@ -925,6 +1005,40 @@ class sock_fprog(ctypes.Structure):
# -- BEST-EFFORT seccomp filter, installed LAST so the manifest reports the truth ---
manifest["seccomp"] = install_seccomp()

# -- CLOSE THE PINNING FD BEFORE THE UNTRUSTED CODE RUNS (wave S0) -----------
# `ws_fd` is the O_PATH handle the PARENT opened on the declared subtree, and it is what made
# the swap detector above a real identity check (stat(target) vs fstat(ws_fd)). Its work is
# finished the moment the bind is verified — and leaving it open was escape B:
#
# * it is inherited by everything below, including the untrusted implementation;
# * its NUMBER is visible, because `sys.argv[4]` is readable from inside the effect;
# * it cannot be `fchdir`'d (O_PATH), but it IS a valid `openat` dirfd against the
# ORIGINAL mount — so `openat(ws_fd, "../..")` reached above the operator's containment
# root, and writes through it succeeded against a subtree mounted MS_RDONLY, whose
# read-only posture had read back correctly.
#
# No syscall filter can fix that: seccomp cannot tell where a dirfd points. The fix is that
# the descriptor stops existing before any untrusted byte runs, and `allowed_fds` stops
# listing it so the fd-closure audit below re-verifies exactly that.
if ws_fd >= 0:
try:
os.close(ws_fd)
except OSError as e:
fatal("workspace pin fd close failed: %s" % e)
allowed_fds.discard(ws_fd)
manifest["workspace_bind"]["pin_fd_closed"] = True
# Read it back: an fd we believe is closed but is not would be the whole escape again,
# so this is verified rather than assumed — the same discipline as every other layer.
try:
os.fstat(ws_fd)
fatal("workspace pin fd still open after close")
except OSError:
pass
# Note there is deliberately no second /proc/self/fd sweep here: after the pivot there is no
# /proc in the jail (nothing mounts one), which is itself part of the containment. The fd
# audit ran earlier, against `allowed_fds`, while /proc was still readable — and the fstat
# read-back above is what proves this particular descriptor is gone.

# -- hand off the honest manifest BEFORE running the effect ------------------
os.write(manifest_fd, json.dumps(manifest).encode())
os.close(manifest_fd)
Expand Down
28 changes: 16 additions & 12 deletions docs/architecture/worker-containment.md
Original file line number Diff line number Diff line change
Expand Up @@ -122,19 +122,23 @@ from inside, and two of the rows above can be. Both are reproduced against a rea
language. They are recorded here so the table is not read as a containment guarantee it does
not deliver.

- **Filesystem isolation — the `chroot` is escapable, on every arch and every profile.**
`chroot()` does not move the cwd, and the worker holds `CAP_SYS_CHROOT` over the user
namespace it was placed in, so a second `chroot()` into a subdirectory plus a `..` walk
re-roots it at the real filesystem root: host files readable *and writable*, as the
parent's real uid. The three adversarial tests in this row assert a genuine refusal on a
genuine path — but none of them attempts to *leave*, so the property a reader infers is
not the property proven. The seccomp filter does not help: `chroot` is not on its denylist.
- **Workspace bind-mount — the pinned `O_PATH` fd outlives the mount setup.** It is passed as
- **CLOSED in wave S0 — filesystem isolation used `chroot`, which was escapable on every arch
and every profile.** `chroot()` does not move the cwd, and the worker holds `CAP_SYS_CHROOT`
over the user namespace it was placed in, so a second `chroot()` into a subdirectory plus a
`..` walk re-rooted it at the real filesystem root: host files readable *and writable*, as
the parent's real uid. The jail now performs `pivot_root` and detaches the old root
(`umount2(MNT_DETACH)`), so the host tree is absent from the worker's mount namespace rather
than merely stepped over — the hostile sequence still runs and resolves inside the jail. The
three adversarial tests in this row assert a genuine refusal on a genuine path but none of
them attempts to *leave*; `tests/adversarial/test_jail_escape.py` is the suite that does,
and it exists because that gap is what let this ship.
- **CLOSED in wave S0 — the pinned `O_PATH` fd outlived the mount setup.** It was passed as
`argv[4]`, reachable from the untrusted implementation via `sys.argv`, and usable as an
`openat` `dirfd` against the *original* mount — so a read-only workspace is writable
through the fd, and `openat(fd, "../..")` reaches above the operator's containment root.
The `statvfs` posture read-back in this row is honest and correct; it is simply not the
only path to those bytes.
`openat` `dirfd` against the *original* mount — so a read-only workspace was writable
through the fd, and `openat(fd, "../..")` reached above the operator's containment root. The
`statvfs` posture read-back in this row was honest and correct; it was simply not the only
path to those bytes. The fd is now closed before any untrusted byte runs, with the closure
read back via `fstat`.

### Authority-seam gates (before any child spawns)

Expand Down
Loading