Skip to content

perf(dsv4): fetch DP prefix caches without source requests - #1611

Open
WANDY666 wants to merge 1 commit into
support_dsv4_modelfrom
perf/dsv4-dp-cache-fetch
Open

WANDY666 wants to merge 1 commit into
support_dsv4_modelfrom
perf/dsv4-dp-cache-fetch

Conversation

@WANDY666

Copy link
Copy Markdown
Contributor

Depends on #1593; targets support_dsv4_model to keep the diff limited to this optimization.

Probe and pin source radix/checkpoints without creating InferReqs or restoring source SWA. Copy compressed history via P2P, transfer checkpoints through Gloo, and restore target state using the existing request manager. Preserve intermediate checkpoints for later radix insertion, Vision boundaries and TP agreement; fence all ranks before releasing source pins.

Avoid duplicate owner probes and the unused shared request table, use per-DP request capacity, and send slot indexes as int32. Generic transfers are unchanged.

Tests: 79 passed, including two-process Gloo transport. One existing flush test fails identically on the base and was excluded. GPU P2P/service validation remains pending.

@WANDY666
WANDY666 requested a review from shihaobai October 8, 2026 03:14
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants