Skip to content

Hosted relay storage is off until an account is approved - #2509

Merged
AbirAbbas merged 18 commits into
devfrom
relay-storage-optin
Nov 8, 2026
Merged

AbirAbbas merged 18 commits into
devfrom
relay-storage-optin

Conversation

@AbirAbbas

@AbirAbbas AbirAbbas commented Nov 8, 2026 •

Copy link
Copy Markdown
Collaborator

Why

The privacy terms for the hosted relay are not finished. Until they are, the hosted relay must not keep anyone's chats or files, not even encrypted. People should still be able to pair their computers and reach one from another while both are on. Relay storage becomes something a person asks for, and we switch it on per account.

What changes

Hosted relay (relay/hosted)

  • Off by default. Every write into relay storage refuses with 403 storage_off, unless the account has been approved or CAF_LIMITS sets storageOpen. It refuses before anything is written and before any quota changes. All such writes go through one place, and a structural test fails if a new one appears.
  • Unchanged: reads, frames already stored, pairing, the device list, presence, the live tunnel behind --at, erasure and rotation.
  • Approval. POST /v1/admin/storage takes an approval signed with an operator key listed in CAF_ADMIN_KEYS. It is bound to the relay host it was sent to, valid for ten minutes, and ordered by its timestamp, so it cannot be replayed or reordered. The public key is committed in wrangler.toml; the private key is not in the repository.
  • Stats now say whether the calling account has storage: storage: "on" | "off".
  • Rotation keeps an approval. A separate, device-signed POST /v1/store/handoff lets the device that froze an identity pass its approval to exactly one successor. The successor proves it holds the successor key. Only on is ever copied, and never over a record the successor already has.
  • Untouched: the shared coordination state machine (internal/clusterstate) and the committed rotation command are byte-identical to dev.

Client

  • Refusal. storage_off is its own refusal, with one plain sentence pointing at Settings.

  • No uploads while off. The client keeps the relay's last answer beside the saved relay and makes no store writes while that answer is off. Chats keep saving on the computer.

  • Discovering approval.

    • One refresher per process (the chat window, codeaf serve, the engine) asks at start, after /status, and every ten minutes while storage is off.
    • It asks nothing while storage is on, and asks a relay that reports no storage state once per process.
    • A chat waiting on storage retries against the local answer at most once a minute, so an approval starts sync within about a minute and needs no restart.
  • Settings → Devices gets a "sync storage" row, named after the "sync service" the Devices tab already shows. When storage is off, enter opens a pre-filled GitHub issue carrying the request code (the account id). The panel also shows the code and the link, for when no browser opens. The row is absent for a relay that reports no storage state, such as the self-hosted one.

  • /status says sync storage off.

  • Quit receipt. Quitting after a revoke says the turn stays on this computer, rather than blaming the relay's reachability.

  • Sync sentences reach the window. On the default road the chat lives in the engine process. The drive's sentences (storage off, storage full, removed, replaced, clock skew, displacement) used to be said on a desk there that no window reads, which was already true on dev. They now travel on each conversation's own notice road, which the engine already forwards to every attached window. A new window is told only a refusal that is still in force.

  • Continuing a chat while storage is off. This is the ordinary road after any --at session, because live access moves the chat's hold to the other computer. The road depends on whether this computer has turns of its own:

    • None: the unchanged path downloads the stored history. Reading needs no approval.
    • Its turns extend the recorded head: it continues in place with no upload, and sends at once.
    • Its turns diverged: it refuses in one sentence and keeps every turn.

    After a later approval, this computer's own expired hold re-acquires and publishes the kept turns.

  • Closing one chat whose turns are saved here says so, instead of "could not be saved".

  • Rotation sends the same freeze bytes as before to every relay. It sends the handoff only to a relay that reports storage. A rotation with storage off finishes with no store writes.

  • codeaf-relay-admin has approve, revoke and public-key. It is a separate program, not part of any release archive or installer.

Your own relay (codeaf relay) is unaffected: storage stays on, with no request step.

Manual. Every section that said chats, keys, teams or task copies travel between paired computers automatically now says this waits for approval on the hosted relay. A new section answers how to request it. The system prompt is unchanged from dev; the manual carries all of this.

Validation

  • Focused tests for every contract item, all passing:

    • the relay's own tests through the real Worker, including the two-device handoff race and live tunnel bytes with storage off;
    • the client gate, refresher, waiter, rotation and handoff;
    • the old freeze bytes against the unchanged strict decoder;
    • the Settings frames, through both the tab and the search, with a failing browser opener;
    • the quit receipt;
    • the manual probes.
  • Measured cost: with storage off, 2–3 local file reads a minute whatever the number of open chats. With storage on, or on a self-hosted relay, nothing.

  • Hand test of the real bin/codeaf against a local copy of the hosted relay, in tmux, with throwaway homes, a throwaway admin key and a keyless model stub:

    • pairing and --at work with storage off, with no store writes;
    • the Settings row, request link and /status line appear;
    • approving without a restart uploads the pending turns, and a headless serve home finds the approval at startup;
    • revoke, replay, wrong key and wrong host are refused;
    • storageOpen opens storage for everyone, and the self-hosted relay is unaffected;
    • rotation of approved and unapproved accounts works;
    • a 10-turn session with quit and relaunch was stable.
  • A real macOS hand check of this branch on a Mac, against a local wrangler dev copy of the hosted relay, on the default engine road. The storage gate held. It found two real bugs and four small ones, all fixed in this branch:

    • the storage-off sentence never reached the window;
    • "continue here" failed after a revoke;
    • closing a chat said it could not be saved;
    • the first /status after pairing lacked the line;
    • the search offered "request access" while storage was on;
    • an open window missed a pairing.

    Mac recheck on the final head 6e031be20: passed.

    • Storage-off continue here after --at asks once, then never again across idle periods.
    • The storage-off sentence shows once per window.
    • No store write succeeds while off.
    • A computer missing newer turns from the other one still asks.
    • Approval publishes the kept turns.
    • Storage-on turns upload normally.
    • make test-touched on macOS attributes nothing to this branch.
    • The relay's own tests pass, 262 of 262.

    An earlier recheck at e91f33289 found one more bug: the continue question returned every few minutes for a chat with turns past the last stored one. It is fixed in 6e031be20.

  • Five read-only reviews. Their findings were fixed in this branch, except the follow-ups listed below.

  • make pr-ready passes. Along the way it caught a lock without defer, an environment read outside internal/env, the device pages' vocabulary law, two contract changes in existing tests (team sharing needs storage; a revoked serving home makes one request), and a refresher check that could outlive its caller. All are fixed here.

Two problems found during the hand test reproduce on dev without this change:

  • continuing a chat on another computer fails on a missing .cell/meta.json;
  • GET /v2/cluster/control can answer 503 to a newly paired identity once several identities are active on a local Worker, until it restarts.

Neither is fixed here.

Known limits, left as follow-ups:

  • Tools in a storage-off chat do not check who holds it. A turn already running on one computer can keep running tools after another computer takes the chat. This was already true for any storage-off chat before this change, and fixing it needs a new check at the tool boundary.
  • Messages from the final save after a chat closes never reach the window. As on dev, the engine stops delivering them before that last save.
  • Local wrangler dev on macOS needs two workarounds, both already on dev: Node 22 or newer, and a [build] hook that reads the Linux-only /proc/pressure/io.

Before merging

Merging to dev redeploys the dev relay, which then stores nothing for accounts that are not approved, including dev and local builds. Approve the accounts that should keep syncing with codeaf-relay-admin approve <id> --relay dev right after the deploy. Production relay deploys stay off.

🤖 Generated with Claude Code

AbirAbbas and others added 18 commits November 7, 2026 21:07
The privacy terms for the hosted relay are not finished, so it must not keep
anyone's chats or files yet. Every R2 write now refuses with 403 storage_off
unless the account was approved or CAF_LIMITS.storageOpen is set; reads,
pairing, the directory, presence and the live tunnel are unchanged, and frames
already stored stay readable.

POST /v1/admin/storage takes an approval signed by an operator key from
CAF_ADMIN_KEYS (the public key is committed in wrangler.toml), bound to the
request host, within ten minutes, and ordered by its timestamp. Stats report
the account's storage state. A separate device-signed POST /v1/store/handoff
lets a frozen identity pass an approval to one proven successor; the shared
coordination evaluator and the committed rotation mutation are unchanged.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The client reads storage_off as its own refusal, keeps the last answer beside
the saved relay, and makes no store writes while a hosted relay says off. One
refresher per process asks at start, after /status and every ten minutes while
off, and asks nothing while on or for a self-hosted relay; a halted drive
retries against the local reading, so approval resumes sync within a minute.

Rotation sends the original freeze bytes to every relay and hands the approval
over through the separate route only when the relay reports storage, before any
resealing upload; a rotation with storage off finishes without store writes.
codeaf-relay-admin approves and revokes accounts; it is not part of any release.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Settings > Devices gains a "relay storage" row on a hosted relay: on, or off
with enter to request access, which opens a pre-filled GitHub issue and also
shows the request code and link in the panel when no browser opens. The row is
absent for a self-hosted relay or an unpaired computer. /status says
"relay storage  off" in place of the sync space line, and the quit receipt
after a revoke says the turn stays on this computer instead of blaming the
relay's reachability.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Every manual section that promised chats, keys, teams or task copies travel
between paired computers automatically now says that on the hosted relay this
waits for storage approval, and a new section answers how to request it. The
system prompt no longer promises automatic hosted sync, and the relay README
documents the operator approval and handoff routes.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…s scoped

Only the device whose freeze committed may hand its approval to a successor,
so a lost computer cannot take the binding first. A cancelled rotation keeps
its successor keys only when a hosted handoff was accepted, and reuses them only
for the same source on the same relay; every other cancel behaves as before.
The off-state check reads each file once, and the manual sections this branch
changed are back under the length limit, with the idle-request page saying that
an off account asks once every ten minutes whether it has been approved.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A relay whose stats carry a storage state is treated as gating whatever its
hostname: the Settings row, the /status line, the local gate and the off-state
refresher all follow that answer. A relay that answers without the field, such
as the self-hosted relay, is asked once per process and then left alone. Before
any answer the hosted hostnames stay closed and every other address stays open.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The new relay-storage sections say share and files often enough that the plain
question reached the sync pages instead; a short section in the asker's words
points at /attach.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…hrough env

Both storage-opened paths now copy what they need inside a short locked helper
that unlocks from a defer, so an absorbed panic cannot leave the drive book or a
live drive locked; the admin command reads CODEAF_RELAY_ADMIN_KEY through the
one environment door.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The Settings row, the /status line, the refusal, the quit receipt and the
request issue now say "sync storage", the name the Devices tab's sync service
already uses, and the device pages keep their vocabulary law. The manual's
storage requirement is one full section per page plus one short sentence only
where a section claims something travels between computers, instead of the
same paragraph pasted into almost every section.

The team-pairing test now states its storage reading and covers paired with
storage off. A serving home starts asking about approval only after its first
eligible state round, so a revoked device still makes one request and stops,
and a revoked or deleted answer ends the asking for good.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The last user to stop the refresher now cancels the request in flight and waits
for that check to finish, and a check that starts after everyone has left does
nothing. Before this a serving loop could return while its startup check was
still saving the relay's answer into the home, which a temporary home's cleanup
saw as "directory not empty" and the race detector saw as a read after the test
ended.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The macOS hand check found these on the default engine road, where the
conversation lives in the engine process and the window is attached to it.

Sync refusals (storage off, storage full, removed, replaced, clock skew,
displacement) were said on a desk inside the engine process that no window
reads. They now go through each conversation's own notice road, which the
engine already forwards to every attached window. Registrations retire with
their owning session, an idle in-process window is woken at once, and a new
window is told only a refusal that is still in force.

Continuing a chat while storage is off chooses its road by local work. With
nothing local to keep, the unchanged path reads the stored history. Local turns
that extend the recorded head continue in place without any upload, and the
chat sends at once instead of waiting for a saver that cannot start while off.
Local turns that diverged are refused with every turn kept. After a later
approval, this computer's own expired hold over an empty head re-acquires and
publishes instead of opening a viewer.

Closing one chat whose turns are saved here no longer says it could not be
saved, and an older peer's positive receipt gets the saved-here wording. The
first /status after pairing shows the storage line, Settings search offers
request access only while storage is off, an open window notices pairing, and
Settings reads storage only where it shows it.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The continuing page says that reading stored history needs no approval, that
only turns made here which another computer lacks wait for storage, and that
coming back after --at works. The close, Settings and starting pages match
what the code now says and does. The change entry records that sync refusal
sentences used to stay in the engine process on the default road.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The prompt on dev says nothing about sync, so the paragraph this branch added
retracted no promise. It only cost about 400 bytes on every request and pushed
the lean prefix over its budget. The manual already answers what sync storage
does and how to ask for it.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…wners close

The storage-off road selector now uses the same local-work checks as the
ordinary continue path, so bookkeeping-only seals let newer history download
and an edit made here stays here without any upload. Sending again after this
computer's own hold over an empty head has lapsed re-acquires the chat
quietly instead of asking to continue it.

A seal watch counts the conversations that use it, Home errands included, and
retires only after the last of them has closed locally. A retired watch arms no
more timers, and the window detaches its listener on every exit.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Retiring a watch was meant to end the delivery of drive sentences once no
conversation uses it. Stopping the trailing seal as well went further than dev
ever did: a conversation still finishing its journal after a deletion timeout,
or a Home errand opened later on the same seat, lost the seal of what it wrote.
Retirement now ends only the subscriptions and the window listener, and the
tail behaves exactly as it does on dev.

The change entry also says that the next Enter, not only a reopen, quietly
reacquires a chat under this device's own expired empty-head hold.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
With storage off no writer renews the directory hold, so it lapses about
ninety seconds after each continue. The send gate then re-acquired quietly only
from the same saved turn or an empty head, and a chat with local turns past the
last stored one asked "Continue this chat here?" again every minute or two,
showing the storage-off sentence each time it reopened.

While storage is off the send gate now treats the chat as local, and a free
chat this computer held last re-acquires quietly when every stored turn is
here. The conversation stays open rather than reopening, so nothing is said
twice. A chat missing turns another computer stored still asks, and nothing
changes while storage is on.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant