Skip to content

feat(fork-sync): snapshot sync, git submodules, and AI model fallback - #4424

Merged
Vladimir Morozov (vmoroz) merged 4 commits into
microsoft:mainfrom
vmoroz:fork-sync-snapshot-submodules
Oct 21, 2026
Merged

Vladimir Morozov (vmoroz) merged 4 commits into
microsoft:mainfrom
vmoroz:fork-sync-snapshot-submodules

Conversation

@vmoroz

@vmoroz Vladimir Morozov (vmoroz) commented Oct 21, 2026 •

Copy link
Copy Markdown
Member

Description

Lets fork-sync roll a large, submodule-based upstream such as Chromium on a regular
cadence. Before this change, rolling a vendored Chromium subset from one release tag to the
next failed in several places: release tags are on separate branches and are not ancestors of
each other, new tags never reached the cached clone, tree:0 history fetches took close to an
hour and then failed, content from git submodules was invisible, and every sync re-added files
the fork had deliberately removed. AI merge had also gone stale: the configured Claude setup no
longer worked, and a retired model ID made every hunk fail.

Snapshot sync (fetchMode: "snapshot") — fetches only the base and target commits
(--depth=1) and merges two synthetic commits that contain just the synced scope, the target
parented on the base. The merge base is exactly the last synced commit, so release-tag rolls
work across branches (e.g. 149.0.7827.149 → 156.0.8078.48) without keeping an upstream
history cache. Tags resolve with ls-remote, and PR notes link the upstream range instead of
listing every commit.

Git submodules (submodules) — listed submodules are synced at the SHAs the upstream
commit pins, at both base and target. Their files are grafted into the synced tree, so 3-way
merge, conflicts, .syncignore, and deletions work on them unchanged, and pins can never drift
from upstream's. Optional fallbackUrls cover mirrors that dropped an old commit. URLs from
.gitmodules may not use the local file transport. PR notes list pin moves, URL changes, and
gitlinks that appeared, disappeared, or are not followed.

Pruned forks (keepLocalDeletions) — files the fork removed stay removed, new upstream
files are added, and only files that upstream deleted are deleted locally, so local-only files
are never touched. A newly configured submodule or sparse path is imported in full. fork-sync
also warns about vendored files hidden by .syncignore that would silently stop updating.

AI merge — the GitHub Copilot CLI is now the default provider, and its default model is
auto, so Copilot routes each request and no model ID goes stale. aiMerge.model accepts an
ordered list, and a model the CLI reports as unavailable falls back to the next entry with a
warning. A new reasoningEffort setting (default low, skipped for models that reject it)
keeps the two calls per hunk cheap. Also fixes hunks being rejected as context mismatches when
the conflict is followed by blank lines, e.g. near the end of a file.

Smaller fixes —

  • sparsePaths entries may be files (e.g. LICENSE).
  • History mode fetches a --tag or --commit that git fetch did not bring in, and caps the
    PR-notes commit table at 200 rows.
  • Fetches skip submodule scanning, auto-gc, and commit-graph writing.
  • --continue is safer: it refuses changed options, and the checkpoint is kept until the
    changes are applied.
  • Output decoding no longer corrupts multi-byte paths split across pipe chunks.
  • Paths are canonicalized, so syncs work when the repo is reached through an 8.3 short name,
    a junction or symlink, or a subst drive. Previously every listing failed with "is
    outside repository".
  • Merge branch names stay unique when two syncs run within the same second.
  • The package CLAUDE.md now points to a new AGENTS.md, as at the repo root.

History mode with none of the new options set behaves as before, apart from the fixes above.

Test plan

  • New end-to-end tests run the CLI against local upstream fixtures. They cover the scenarios
    above: release branches, a whole-directory submodule served only by a fallback URL, unfollowed
    and newly appearing gitlinks, prunes, subDir, conflicts with --continue and --abort,
    switching modes, and config validation. Unit tests cover the new git plumbing, PR-notes
    formatting, and model fallback. yarn test passes, and the Copilot integration tests pass
    with RUN_INTEGRATION=1.
  • Tried on a real Chromium subset with 9 followed submodules. Re-syncing the pinned tag is now a
    true no-op; before, it re-added about 3,000 removed files. A roll from 149.0.7827.149 to
    156.0.8078.48 completed the merge with 6 expected conflicts, all in files the fork modifies.
  • To try snapshot mode, set "fetchMode": "snapshot" in a dependency's sync-config.json and
    run fork-sync --dep <name> --tag <tag>. See the README sections "Snapshot mode", "Git
    submodules", "Pruned forks", and "Choosing AI models".

…I model fallback

Make fork-sync able to roll huge, submodule-based upstreams such as Chromium
on a regular cadence:

- fetchMode "snapshot": fetch only the base and target commits (depth 1) and
  merge synthetic snapshots of the synced scope whose merge base is exactly the
  last synced commit, so release tags on different branches roll cleanly.
- submodules: follow listed gitlinks at the SHAs upstream pins, with fallback
  URLs, grafted into the synced tree so merge/conflict/apply work unchanged.
- keepLocalDeletions: pruned files stay pruned, new upstream files are added,
  only upstream deletions are applied; warns about .syncignore-hidden files.
- File entries in sparsePaths; history-mode tag/commit fetch, capped PR-notes
  commit table, fetch hygiene; safer --continue checkpoints.

Keep ai-merge working as AI models change:

- Copilot CLI is the default provider with model "auto" (Copilot routes each
  request) and reasoning effort "low" where supported.
- aiMerge.model accepts an ordered list; unavailable models fall back to the
  next entry with a warning.
- Fix hunks rejected as context mismatches when the context ends with blank
  lines (e.g. a file's final newline).

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Merge branch names used a one-second timestamp, so two syncs of the same
dependency within one second failed with "a branch named ... already exists"
(seen in CI, where the e2e syncs run back to back). Append a numeric suffix
when the name is taken; add a test that occupies upcoming names.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
On fast CI the previous sync's branch for the current second is still
checked out in the sync repo, so `git branch -f` fails. Create only the
missing names; existing ones are already taken.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
git reports the repository root with symlinks, junctions and Windows 8.3
short names resolved, but the manifest and dependency paths came from the
working directory as given. When the two spellings differed (e.g. a TEMP of
C:\Users\RUNNER~1 on CI, a subst drive, or -C through a junction), the
dependency prefix became "../../.." and every git listing failed with "is
outside repository". Resolve the working directory, manifest directory and
repo root with realpath first; add a test that syncs through a junction.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@vmoroz
Vladimir Morozov (vmoroz) merged commit eae664f into microsoft:main Oct 21, 2026
12 checks passed
@vmoroz
Vladimir Morozov (vmoroz) deleted the fork-sync-snapshot-submodules branch October 21, 2026 12:54
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants