Skip to content

wire, sdk/go: streaming consumers decode bulk frames into one reused buffer - #328

Merged
CMGS merged 1 commit into
mainfrom
perf/sdk-transient-decode
Oct 5, 2026
Merged

CMGS merged 1 commit into
mainfrom
perf/sdk-transient-decode

Conversation

@CMGS

@CMGS CMGS commented Oct 5, 2026

Copy link
Copy Markdown
Contributor

What

Every stdout, stderr and data frame used to be base64-decoded into a fresh buffer (256 KiB per bulk frame). So Pull, ReadFileTo, a PortConn, or a Run with streaming output reallocated its whole transfer and spent the GC on it.

API additions (existing behavior is unchanged):

  • wire.DecodeResponseInto(line, &scratch): decodes a canonical bulk frame into a caller-owned buffer, growing it as needed. The returned Data is valid until the next call with the same buffer. DecodeResponse keeps its fresh-buffer contract.
  • silkd.Conn.RecvTransient(): Recv with one scratch buffer per connection. Recv is unchanged.

SDK internals:

  • These readers hand each payload to a sink or io.Writer that does not keep it, so they now read transiently:
    • drainData (ReadFile, ReadFileTo, Pull, PortConn);
    • pumpStdio (Run, Exec, Attach, Logs);
    • the pty drain.
  • ReadFile used to keep the decoded frames; it now appends them into its own buffer. Its old comment ("retaining b is safe: fastBulk decodes each frame into a fresh buffer") is gone.
  • drainData's doc now states the sink contract. io.Writer already forbids retaining p.

wire internals: the bulk fast path moved from a per-tag fastBulk decoder to a bulkResponses table consulted before responseDecoders. Tags now come from each type's RespType(), so there are no repeated literals.

Numbers

Microbench (Apple M-series, BenchmarkDecodeBulk):

frame DecodeResponse DecodeResponseInto
4 KiB 2486 ns, 4888 B/op 2071 ns, 24 B/op
256 KiB 139.2 µs, 270 KB/op 126.6 µs, 52 B/op

End to end on a bare-metal test host: e2e/cmd/pullbench pulling a 512 MiB guest file, 6 pulls per run, the two middle samples shown. Runs were interleaved b9/main/main/b9/b9/main/main/b9.

SDK MiB/s
main 195–196, 184, 229–240, 208–212
this PR 242–258, 230–231, 235–239, 233–248

That is about +16% averaged over the four runs. An earlier hack that reused one global buffer (the upper bound) measured 207–226 vs 247–263 MiB/s, about +18%. The gain is larger than the decode share alone (about 10% in a single-core microbench) because the 256 KiB-per-frame garbage also stops driving GC.

Release note

sdk/go builds against the in-repo protocol/wire through its replace. Downstream users of the next sdk/go tag need a protocol/wire tag that includes DecodeResponseInto, cut in lockstep as every release already does.

Tests

  • TestDecodeResponseIntoReusesScratch: the second frame lands in the same backing array; a non-canonical shape still decodes; malformed frames are rejected.
  • TestDecodeResponseIntoMatchesDecodeResponse: DecodeResponseInto and DecodeResponse agree on every resp_* fixture in the golden corpus.
  • TestRecvKeepsDataWhileRecvTransientReusesIt (silkd): Recv frames stay intact, and RecvTransient reuses its buffer.
  • The existing multi-frame ReadFile test (640 KiB across three frames) checks the new append path.
  • Gates:
    • make go-lint: 10× 0 issues.
    • asl clean for wire and sdk/go on darwin and linux.
    • go test -race for wire, sdk/go, e2e, mcp and sandboxd: passes on macOS and on linux/arm64.

…buffer

Every stdout, stderr and data frame was base64-decoded into a fresh 256 KiB buffer, so a Pull or ReadFileTo allocated its whole size again and spent the GC on it. wire.DecodeResponseInto decodes a bulk frame into a caller-owned scratch buffer, and silkd.Conn.RecvTransient uses one per connection; Recv and DecodeResponse keep their fresh-buffer contract. drainData, pumpStdio and the pty drain hand each payload to a writer or sink that does not keep it, so they read transiently; ReadFile, which kept the frames, now appends them into its own buffer.
@CMGS
CMGS merged commit bed3912 into main Oct 5, 2026
3 checks passed
@CMGS
CMGS deleted the perf/sdk-transient-decode branch October 5, 2026 09:58
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant