fix(oss-commit-sync): skip an export whose content OSS already holds - #275
sydorovdmytro wants to merge 2 commits into
Conversation
The import checks external_is_benign before applying, because a commit whose post-image is already present can only produce a no-op or a spurious conflict. The export had no equivalent, and its post-apply nothing_staged check never runs when git apply itself fails -- which for this shape it does: a patch that deletes a path OSS no longer has cannot apply at all. A release line reaches that state whenever its OSS branch is cut from the default branch after a change landed there, so the monorepo's backport of the same change is the first thing replayed onto a tree that already has the result. On v0.37 all 391 paths the commit touched already matched, 47 of them deletions, and the run died calling it a conflict. Add monorepo_is_benign, the mirror of the import's helper, and check it before applying.
loft-bot
left a comment
There was a problem hiding this comment.
Panel review: 2 blocking, 5 quality notes, 8 lanes. Well-motivated fix with a real reproduction behind it; both blocking items were reproduced locally against the action's own bats fixture.
Blocking concerns
.github/actions/oss-commit-sync/lib.sh:948-950— the comparison reads blobs, not tree entries, so a mode-only commit is judged benign; it mirrored correctly at base and now hard-fails the run at the convergence assertion..github/actions/oss-commit-sync/test/export.bats:1261— the new helper's "deletion OSS still needs" branch is untested; a mutation making every deletion benign keeps all 52 cases green.
What was checked
Checked: correctness, architecture, reuse, test-quality, coverage gaps (analyzer plus independent verifier), operability, typos, PR metadata. Skipped: security (no injection sink, secret, authz or untrusted-input surface), dead-code (zero deletions in the diff), infra (no Terraform, Helm, manifest, workflow or Dockerfile), the four e2e lanes and entry-point fidelity (no e2e suite, and the linked ticket is an acceptance-criteria ticket rather than a customer bug), layout coherence (not a restructure-shaped diff). No sibling open PR touches this action, so no merge-order conflict.
Quality notes (non-blocking)
- consider —
export.sh:435:$(git -C "$WT" rev-parse HEAD)sits in theif's tested command, so errexit is suspended and a failed read passes an empty argument; the helper then resolves":${path}"against the monorepo index and answers benign for a delete-only commit (checked directly: rc 1 with the real tip, rc 0 with an empty argument). Unlikely to fire on a just-created worktree, but it contradicts the helper's own fail-closed comment — capture and check first, as the other producers here do. - consider —
export.bats:1261: this test stays green against baseea06a677with the fix fully reverted, so it does not pin the change; the sibling at 1216 carries the pre-fix-red value. Worth correcting the PR body's "both verified red" claim. This becomes blocking if the sibling test is ever weakened or removed, leaving no red-without-the-fix guard. - consider —
export.bats:1216: the rename decomposition the helper's comment calls out is unexercised; re-adding-Mto thediff-treecall leaves all 52 cases green while reproducing the originaldoes not exist in indexconflict. This becomes blocking if-Mor rename handling in this helper is changed later. - consider —
export.sh:436:Skipping ... (content already on OSS)is a substring of thenothing_stagedmessage below it, so neither the log nor the new test's assertion can tell the two skip paths apart. Wording unique to the pre-apply check would fix both. - consider —
lib.sh:929:monorepo_is_benignmirrorsexternal_is_benign(lib.sh:547) closely enough that a future fix to one can easily miss the other — the mode issue above is an instance. A single parameterized helper taking the two(sha, prefix)pairs would keep them in step.
| blob_mono="$(git rev-parse --quiet --verify "${m}:${SUBTREE_PREFIX}/${path}" 2>/dev/null)" || return 1 | ||
| blob_oss="$(git rev-parse --quiet --verify "${oss}:${path}" 2>/dev/null)" || return 1 | ||
| [ "$blob_mono" = "$blob_oss" ] || return 1 |
There was a problem hiding this comment.
blocking — git rev-parse <rev>:<path> yields the blob, not the tree entry, so the file mode is outside this comparison: a mode-only commit (a chmod +x on a script under the prefix) matches on every path and is judged benign.
Reproduced on this suite's own fixture with npx bats@1.11.0, using one commit that only chmods the seeded pkg/app.go:
- at base
ea06a677it replays and pushes — OSS ends at100755, exit 0; - at this head it is skipped as
content already on OSS, and the run then dies at the convergence assertion:::error::OSS tree does not match the monorepo staging tree after replay: pkg/app.go | 0→Re-run with align-tree=true, exit 1, OSS left at100644.
So a case that mirrored correctly before this change becomes a hard export failure that blocks every later commit on the branch — the same stalled-mirror outage this PR exists to remove — and the only recovery offered is align-tree, which rewrites the whole tree. The helper's "a false benign cannot corrupt the mirror" reasoning does hold (nothing is corrupted), but here the assertion is the failure rather than a safety net behind one.
Comparing the tree entry keeps the mode in scope. Verified with exactly this patch: the mode-only commit replays again, and export.bats (52 cases) plus import.bats (26) stay green.
| blob_mono="$(git rev-parse --quiet --verify "${m}:${SUBTREE_PREFIX}/${path}" 2>/dev/null)" || return 1 | |
| blob_oss="$(git rev-parse --quiet --verify "${oss}:${path}" 2>/dev/null)" || return 1 | |
| [ "$blob_mono" = "$blob_oss" ] || return 1 | |
| blob_mono="$(git ls-tree "$m" -- "${SUBTREE_PREFIX}/${path}")" || return 1 | |
| blob_oss="$(git ls-tree "$oss" -- "$path")" || return 1 | |
| [ -n "$blob_mono" ] && [ "${blob_mono%%$'\t'*}" = "${blob_oss%%$'\t'*}" ] || return 1 |
external_is_benign compares blobs the same way, but there the verdict only suppresses a guard complaint; here it decides whether a commit is replayed at all, so the two are not equivalent risks — worth fixing both if you touch the shared shape.
| [[ "$output" != *"conflict replaying"* ]] | ||
| } | ||
|
|
||
| @test "a commit that still has something to add is not skipped as benign" { |
There was a problem hiding this comment.
blocking — DEVOPS-1545's third criterion ("a commit that still has something to contribute is NOT skipped") is covered here only for the add/modify shape. The D-status branch is the one piece of bespoke logic in the new helper (git cat-file -e "${oss}:${path}" → not benign), and no spec drives a monorepo commit that deletes a path OSS still holds — git rm appears only in the sibling test at line 1216, where OSS has already lost the path.
Confirmed by mutation: replacing that branch's return 1 with a bare continue, so every deletion counts as benign, leaves all 52 export.bats cases green. A probe then shows the mutant skips the real deletion and dies later at the convergence assertion (OSS tree does not match the monorepo staging tree after replay: pkg/keep.go | 1 -), while this head replays it correctly.
Nothing is silently lost — the convergence assertion does catch it, so the ticket's claim on that point holds. But it converts every deletion-carrying export into a hard-failed run needing manual align-tree, which is the same outage class the ticket was filed for, and it would ship green.
Suggested case, mirroring the one at line 1216 but without the upstream deletion: export pkg/keep.go, git rm it in the monorepo only, then assert the run exits 0, the output does not contain content already on OSS, and git -C "$OSS_REMOTE" cat-file -e main:pkg/keep.go fails (the deletion was mirrored).
…enign `<rev>:<path>` resolves to content only, so a mode-only change was invisible: a chmod +x under the prefix matched on every path, the commit was skipped as already mirrored, and the run then died at the convergence assertion over the mode it had just ignored -- turning a case that mirrored correctly into a stalled branch whose only offered recovery was align-tree. Both directions ask the same question of different pairs, so the comparison now lives in one helper and the fix lands on both. Read the OSS tip outside the `if` as well: a command substitution in a tested condition has errexit suspended, and an empty rev resolves against the index and answers "benign" for a delete-only commit. Cover the deletion branch, the mode case and renames. Verified by mutation: making every deletion benign, comparing blobs instead of entries, and dropping the pre-check each turn a test red.
|
Both blocking items were real and are fixed in f7e0742. I reproduced each before changing anything.
Took your point about fixing both. The comparison now lives in one
On
On the PR body's "both verified red". You were right to call that out, and I should have checked rather than asserted it. Only the first of the two pinned the change; the second guards against over-skipping and passes either way. I have since mutation-tested every new assertion individually — deletion-always-benign, blob-instead-of-entry, 171/171, shellcheck clean. |
The import checks
external_is_benignbefore applying, with the reasoning in its comment: a commit whose post-image is already present "can only produce a no-op or a spurious conflict". The export had no equivalent — only the post-applynothing_stagedcheck, which never runs whengit applyitself fails.For this shape it does fail. A patch that deletes a path OSS no longer has cannot apply; there is nothing to 3-way against.
A release line lands in that state whenever its OSS branch is cut from the default branch after a change reached it, so the monorepo release branch carries a backport of that same change and the backport is the first thing replayed. On
vcluster-prov0.37the anchor was#2321onmain, the first replayed commit was#2322(itsv0.37backport), all 391 touched paths already matched its post-image, 47 of them deletions — and the run reported a conflict on a commit with nothing to contribute.Closes DEVOPS-1545
Test plan
export.batscases: already-mirrored commit skipped rather than called a conflict; a commit with something to add not skipped; a deletion OSS has not seen still mirrored; a mode-only change still mirrored; a rename decomposed and mirroredv0.37export gets past6a7b08b3and proceeds. It then stops on a genuinepkg/platform/version.goconflict, which is a separate problem being handled by hand in chore: replay the pending v0.37 commits from vcluster-pro vcluster#4183