review: sandbox on encoding/json/v2

[CMGS]

Summary

Every module (protocol/wire, sandboxd, sdk/go, e2e, mcp) now calls encoding/json/v2 and encoding/json/jsontext. The one remaining encoding/json import supplies DefaultOptionsV1() for the two digests that must keep their v1 bytes.

Kept as before

• ClusterDigest and the pools.json seed hash marshal with the v1 option set, so a rolling upgrade keeps the mesh digest and the warm pools. Golden tests pin the v1 bytes (TestClusterDigestMatchesTheV1Bytes, TestPoolSeedHashMatchesTheV1Bytes).
• Request bodies and MCP input match member names case-insensitively (TestClaimMatchesBodyKeysCaseInsensitively, TestRenewMatchesBodyKeysCaseInsensitively).
• Guest bytes that are not valid UTF-8 come out as U+FFFD in sandboxd responses, the usage and audit journals, and MCP replies (TestExecKeepsInvalidUTF8OutputAsReplacementCharacters, TestJournalKeepsAnEventWithInvalidUTF8, TestToolRepliesReplaceInvalidUTF8).
• MCP input with invalid UTF-8 or a lone surrogate is decoded the same way; a plain v2 decode drops the line without a reply (TestServeSpeaksMCP).
• MCP replies sort map keys, so tools/list is byte-stable across calls and sessions (TestRepliesEncodeMapsInSortedOrder).
• A renew without a body takes the node default.
• A nil stdin or upload chunk encodes as "". That is now the v2 default, so wire.B64 is gone (TestRequestDataNeverNull).
• The MCP ping reply keeps its empty result object (result,omitzero).

Changed on purpose

• Nil slices and maps encode as [] and {}. silkd's Vec fields accept both. A git add or replace with an empty list is now a no-op instead of bad_request, which is what an explicit empty list already did.
• Map keys are no longer sorted in sandboxd responses, journals and gossip. The digests and MCP replies still sort them.
• A request body with invalid UTF-8 gets a 400. So does one with data after the value, except renew's optional body, which stops at the first value.
• Case-insensitive matching also ignores - and _. DecodeStrictJSON already did this on main.
• JSON responses no longer end with a newline.
• sdk/go returns an error for an argv or path that is not valid UTF-8, instead of sending U+FFFD.

Hot path

Request bodies decode through UnmarshalRead and responses encode through MarshalWrite. This is an interleaved A/B with the order swapped, on a claim-shaped payload:

	v1 (main)	v2, fresh jsontext coder	v2 (this PR)
decode a claim body	1020 ns, 1049 B, 11 allocs	1250 ns, 6032 B, 21 allocs	670 ns, 480 B, 6 allocs
encode a response	323 ns, 160 B, 2 allocs	535 ns, 800 B, 11 allocs	342 ns, 160 B, 2 allocs

decodeOptionalBody (renew only) keeps a jsontext.Decoder, because it is the only reader that tells an empty body (io.EOF) apart from a truncated one.

Review

• The four /simplify lenses (reuse, simplification, efficiency, altitude) ran over the diff.
  • Applied: dropping wire.B64, pooled body decode and response encode, MCP replies keeping invalid guest UTF-8, and digests on DefaultOptionsV1. The plain v2 swap had broken the MCP replies: a read of a binary file made serve exit.
  • Not applied: jsontext.Decoder.SkipValue for the tag scan. It would raise the nesting bound from 16 to 10000, and a hostile frame would then cost about 470 KB of heap per scan (measured).
• Five full-file readers ran: style over all 52 changed or due files, and judge over the 25 production files.
  • Applied from them: sorted map keys in MCP replies, so tools/list stays byte-stable for a host's prompt cache (v2 gave 8 encodings of one schema in 50 runs). MCP input with a lone surrogate or invalid UTF-8 is now decoded with U+FFFD; before, the line was dropped without a reply. DigestJSON also gains a one-line godoc.
  • Rejected: a claim that lowercase inline comments break the register (the register is lowercase-first), and a claim that the commit body was duplicated (it is not).
  • Kept, with a reason in the hygiene ledger:
    • The tag scan's own 16-deep bound.
    • The audit decode dropping a hand-built relay frame that has duplicate member names. This is contrived: it needs the raw relay, v1 already had the same class of hole for a wrong-typed field, and docs/security.md places callers inside the trust boundary.

Gates

Run on 2899388:

• make go-lint: rc=0, 10 of 10 0 issues. lines (5 modules, linux and darwin).
• asl -forwarder=false ./...: 0 findings in every module, on both GOOS.
• make go-test (race): 20 ok, 0 FAIL.
• linux/arm64 golang:1.27.1 container, uid 1000: protocol/wire, sdk/go, sandboxd, mcp and e2e all ok.
• Every new test was mutation-checked: dropping the option it pins makes it fail, three runs out of three for the MCP ordering and input tests.

Hardware (host A, linux/amd64, fork-dev CH)

Frozen set, sha-checked before and after every lane:

• cocoon master a93295c
• vk-sandbox master 93413a4
• sandbox-operator master 0c5b852
• agent-sandbox ce66bdc
• template sandbox/rt:24.04 = sha256:a29e0acd
• the main arm is sandbox 9118b04

lane	what	result
SBX	Pod path (upstream controller + vk-sandbox), L3 aggregated API, e2b surface, envd-proxy: the older v1 SDK in vk-sandbox and sandbox-operator against the new sandboxd	28/28 PASS
SBS-01	scripts/sandboxd-e2e.sh: golden, warm pool, claim tiers, the SDK smoke (files, session, find/replace, watch, git, pty), reap, restart	PASS on v2 (21.2 s) and on main (21.3 s)
SBS-02	scripts/archive-e2e.sh: idle hibernate, store archive, wake with the same id and guest state	PASS on v2 (41.2 s) and on main (41.3 s)
SBU	in-place main -> v2 -> main on one data dir	7/7 PASS
SBM	the real MCP binary over stdio against a live sandboxd	PASS on v2 and on main

SBU in detail:

• The main-era claim execs and renews on v2, and the main checkpoint restores its guest state there.
• After the rollback, main restores the checkpoint that v2 took.
• pools.json is adopted in both directions with 0 seed-mismatch warnings.
• A control run shows the warning check is live: the same pools.json under a changed config warns once.

SBM in detail:

• exec, read_file and spawn/logs turn invalid guest bytes into U+FFFD.
• tools/list is byte-identical across calls: 9149 bytes on both arms.
• A lone-surrogate request gets a reply.
• The server exits 0 on stdin EOF.

SBX, SBS and SBU ran the sandboxd build of 03e9da0. The final head 2899388 changes only utils.go's godoc and import spelling for sandboxd, and a -buildvcs=false -buildid= build of both commits has identical .text, .rodata and .data; only the .gopclntab line table differs. SBM ran the 2899388 build.

Size

• prod +118/-114, net +4
• tests +277/-127, net +150
• comment lines +1/-1