Repository navigation
wire, sdk/go: streaming consumers decode bulk frames into one reused buffer - #328
Merged
Merged
Conversation
…buffer Every stdout, stderr and data frame was base64-decoded into a fresh 256 KiB buffer, so a Pull or ReadFileTo allocated its whole size again and spent the GC on it. wire.DecodeResponseInto decodes a bulk frame into a caller-owned scratch buffer, and silkd.Conn.RecvTransient uses one per connection; Recv and DecodeResponse keep their fresh-buffer contract. drainData, pumpStdio and the pty drain hand each payload to a writer or sink that does not keep it, so they read transiently; ReadFile, which kept the frames, now appends them into its own buffer.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Every
stdout,stderranddataframe used to be base64-decoded into a fresh buffer (256 KiB per bulk frame). SoPull,ReadFileTo, aPortConn, or aRunwith streaming output reallocated its whole transfer and spent the GC on it.API additions (existing behavior is unchanged):
wire.DecodeResponseInto(line, &scratch): decodes a canonical bulk frame into a caller-owned buffer, growing it as needed. The returnedDatais valid until the next call with the same buffer.DecodeResponsekeeps its fresh-buffer contract.silkd.Conn.RecvTransient():Recvwith one scratch buffer per connection.Recvis unchanged.SDK internals:
io.Writerthat does not keep it, so they now read transiently:drainData(ReadFile,ReadFileTo,Pull,PortConn);pumpStdio(Run,Exec,Attach,Logs);ReadFileused to keep the decoded frames; it now appends them into its own buffer. Its old comment ("retaining b is safe: fastBulk decodes each frame into a fresh buffer") is gone.drainData's doc now states the sink contract.io.Writeralready forbids retainingp.wire internals: the bulk fast path moved from a per-tag
fastBulkdecoder to abulkResponsestable consulted beforeresponseDecoders. Tags now come from each type'sRespType(), so there are no repeated literals.Numbers
Microbench (Apple M-series,
BenchmarkDecodeBulk):DecodeResponseDecodeResponseIntoEnd to end on a bare-metal test host:
e2e/cmd/pullbenchpulling a 512 MiB guest file, 6 pulls per run, the two middle samples shown. Runs were interleaved b9/main/main/b9/b9/main/main/b9.That is about +16% averaged over the four runs. An earlier hack that reused one global buffer (the upper bound) measured 207–226 vs 247–263 MiB/s, about +18%. The gain is larger than the decode share alone (about 10% in a single-core microbench) because the 256 KiB-per-frame garbage also stops driving GC.
Release note
sdk/gobuilds against the in-repoprotocol/wirethrough its replace. Downstream users of the nextsdk/gotag need aprotocol/wiretag that includesDecodeResponseInto, cut in lockstep as every release already does.Tests
TestDecodeResponseIntoReusesScratch: the second frame lands in the same backing array; a non-canonical shape still decodes; malformed frames are rejected.TestDecodeResponseIntoMatchesDecodeResponse:DecodeResponseIntoandDecodeResponseagree on everyresp_*fixture in the golden corpus.TestRecvKeepsDataWhileRecvTransientReusesIt(silkd):Recvframes stay intact, andRecvTransientreuses its buffer.ReadFiletest (640 KiB across three frames) checks the new append path.make go-lint: 10×0 issues.aslclean for wire and sdk/go on darwin and linux.go test -racefor wire, sdk/go, e2e, mcp and sandboxd: passes on macOS and on linux/arm64.