Repository navigation
fix(ops): upload the R2 weekly archive as multipart parts, not one PUT - #1911
Merged
Merged
Conversation
The weekly archive (~1.5 GB gzip) went up as a single signed PUT. Twice on 2026-11-07 the transfer died mid-flight with UND_ERR_SOCKET after several minutes and the run reported upload_failed — snapshot, verification, and compression all succeeded; only the single-shot transfer failed, and one dropped connection voided the whole upload. Replace it with an S3 multipart upload built on the existing hand-rolled signer: CreateMultipartUpload carries the sha256/source-bytes/created-at metadata, parts of ~64 MB are read from the gzip file one at a time (the file is stream-hashed first; it is never loaded into memory), each part is retried up to 3 times, CompleteMultipartUpload sends the part ETag list, and AbortMultipartUpload runs on any unrecoverable failure. A 200-with-<Error> completion body is treated as the failure it is. Failure messages keep the S3 PUT prefix so classifyFailure still reports upload_failed. Unchanged: env vars, bucket/prefix/key naming, status-file shape and reasons, the download-back verification (hash + gunzip + integrity_check + embedded sha), prune behavior, the kill switch, and exit codes. The download-back GET stays a single streamed read — splitting it is a separate change if it ever becomes the failure point. Tests: the in-memory S3 stub learns multipart (create/parts/complete/abort); the upload-transport-rejection test now expects create + 3 part attempts + abort; three new uploadMultipart tests cover multi-part assembly, retry of a dropped part, and abort with the signer error preserved. npm run test:r2-archive: 28 passed. Refs UM-1538.
Code Review Completed! 🔥The code review was successfully completed based on your current configurations. Kody Guide: Usage and ConfigurationInteracting with Kody
Current Kody ConfigurationReview OptionsThe following review options are enabled or disabled:
|
![kody-ai[bot]](https://avatars.githubusercontent.com/in/413034?s=60&v=4){kind=small}
uploadMultipart never validated partBytes: 0 makes partCount Infinity and the part loop issues empty PUTs forever instead of failing. Validate partBytes as a positive integer and fileBytes as a non-negative integer before CreateMultipartUpload, with a test asserting no S3 call happens. 29 tests pass.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Replaces the single-PUT R2 weekly archive upload with S3 multipart upload using ~64 MB parts. The change is motivated by two mid-transfer failures on 2026-11-07 where a dropped connection voided an ~1.5 GB transfer that had to restart from byte zero. Each part is individually signed and retried up to 3 times, so a dropped connection costs one part rather than the full object, and an unrecoverable failure issues AbortMultipartUpload so no orphan parts linger. The archive flow now hashes the gzip via streaming and reads parts from disk, avoiding holding the full gzip in memory, while preserving metadata on CreateMultipartUpload and the existing download-back verify of re-hash, decompress, and integrity_check before pruning. Create and complete failures are remapped to preserve the prior
upload_failedclassification, and a 200 response carrying an S3<Error>body on CompleteMultipartUpload is treated as failure. Tests added cover sequential part assembly, retry after a dropped connection, and abort preserving the signer error, with the in-memory S3 stub extended for multipart lifecycle; live transfer behavior remains unverified here. Automated review reports one unresolved medium finding: invalidpartBytescauses an infinite multipart upload loop inuploadMultipart.