Skip to content

Price dense recompute by its traced stage and one input gradient #986

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Draft
bradhilton wants to merge 19 commits into
base: dalinar/cp-layout-recompute-floor
Choose a base branch
from
Draft

Price dense recompute by its traced stage and one input gradient #986

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
19 commits
Select commit Hold shift + click to select a range
29024e3
Price dense recompute by its traced stage and one input gradient
bradhilton Sep 26, 2026
b8bc47e
Charge dense no-grad transients and recompile residue; harden the gate
bradhilton Sep 26, 2026
bf98ce8
Decide dense eligibility once per wave; check wrapper delegates
bradhilton Sep 26, 2026
0518873
Accept Dynamo-compiled GDN layer delegates in the dense gate
bradhilton Sep 26, 2026
7ff82bc
Match the dense gate to the traced Qwen execution
bradhilton Sep 26, 2026
da1294d
Check everything a dense layer runs before pricing its traced stage
bradhilton Sep 26, 2026
c9d63e2
Merge the rebased per-rank CP2 layouts into dense recompute pricing
bradhilton Sep 26, 2026
9fe8daf
Merge the per-rank CP2 layout review fixes into dense recompute pricing
bradhilton Sep 26, 2026
865eb6a
Test that dense widths stay at TP1 where a sequence-parallel floor pr…
bradhilton Sep 26, 2026
c5b4d2a
Merge the layout-path TE test into dense recompute pricing
bradhilton Sep 27, 2026
b7f6dc3
Merge the layout-boundary adapter-gradient port (#978 53feda89b)
bradhilton Sep 28, 2026
1105078
Merge the #978 combine-floor test (5163520d8)
bradhilton Sep 28, 2026
cc4c777
Merge the #978 per-rank adapter pairing (75a3912a7)
bradhilton Sep 28, 2026
5166260
Merge #978's sequential gradient groups (f47a06f27)
bradhilton Sep 28, 2026
8b2a46c
Merge #978's pending-gradient tests (29c6a82ad)
bradhilton Sep 28, 2026
f912511
Merge #978's closed-form group order and pending reuse (500553bb4)
bradhilton Sep 28, 2026
f57fbca
Merge #978's two-slot split test (6eb86f7b4)
bradhilton Sep 28, 2026
2e03fe3
Merge #978's two-slot lower-bound assertion (bd26e7e9a)
bradhilton Sep 28, 2026
0c0e3fc
Merge #978's layer-mismatch test
bradhilton Sep 28, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 3 additions & 1 deletion .github/workflows/prek.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -244,6 +244,7 @@ jobs:
tests/unit/test_trainer_rank_converted_memory.py \
tests/unit/test_trainer_rank_layout_memory.py \
tests/unit/test_context_parallel_retained_bytes.py \
tests/unit/test_trainer_rank_dense_memory.py \
tests/unit/test_trainer_rank_split.py \
tests/unit/test_megatron_compile_garbage.py \
tests/unit/test_trainer_rank_cache_recovery.py::test_dense_cp_exact_demand_fits_after_recovery \
Expand Down Expand Up @@ -293,4 +294,5 @@ jobs:
--ignore=tests/unit/test_megatron_compile_garbage.py \
--ignore=tests/unit/test_trainer_rank_converted_memory.py \
--ignore=tests/unit/test_trainer_rank_layout_memory.py \
--ignore=tests/unit/test_context_parallel_retained_bytes.py
--ignore=tests/unit/test_context_parallel_retained_bytes.py \
--ignore=tests/unit/test_trainer_rank_dense_memory.py
Loading
Loading