Repository navigation
Fix vmem_chunked allocator alignment, granularity, and rollback edge cases - #633
Open
Abhishek-Varma wants to merge 1 commit into
Open
Abhishek-Varma wants to merge 1 commit into
Abhishek-Varma wants to merge 1 commit into
Conversation
Addresses the bug-fix bundle from the vmem_chunked hardening epic. Scope is AMD-only: the shared driver interface change is implemented and tested on the AMD local and fabric drivers; NVIDIA keeps its existing behavior via the base-class default (no untested driver code is shipped). 1. Granularity reporting: distinguish minimum vs recommended allocation granularity. Add BaseDriver.get_recommended_granularity() (a concrete default returning the minimum) and query both minimum/recommended from the AMD local and fabric drivers. The allocator now backs memory in units of the recommended granule while reporting the true minimum via get_stats() (recommended_granularity / min_granularity). AMD behavior is byte-identical since the old get_minimum_granularity() already returned the recommended value; NVIDIA inherits the default (recommended == minimum), i.e. unchanged. 2. Alignment: honor a requested alignment larger than the granule. allocate() computes an effective alignment (>= min_alignment, rounded to a power of two) and aligns the heap offset to it, keeping allocations symmetric across ranks. Free lists are keyed by (size, alignment) -- a "Bucket" -- so a block is only reused for a request with matching alignment. 3. Use-after-close: add _check_open() guards to allocate(), import_external_tensor(), import_peer_chunk() and get_allocation_chunks_since(); operations after close() now raise RuntimeError instead of corrupting state. 4. Failed-allocation rollback: build the tensor before committing bump pointer and bookkeeping. If tensor construction raises, a popped free-list block is returned and the bump pointer is left untouched -- no orphaned reservation, so a rank can always re-converge to the symmetric layout. 5. Non-power-of-two heap sizes: drop the power-of-two chunk_size requirement. VA reservation is rounded up to a whole number of chunks arithmetically (math.ceil) instead of via a bitmask, so arbitrary granularity-aligned chunk sizes work for any heap size. Adds reproduce-first unit tests for all five fixes. Full allocator suite passes 37/37 on 2x MI300A (1- and 2-rank). Review follow-ups folded into this commit: - Renamed the HIP _query_granularity(flag) parameter to granularity_flag and documented it, instead of a bare, unexplained "flag". - Named and documented the (size, alignment) free-list key as a module-level Bucket type alias, with a defining comment at every site that creates, stores, or reads one (previously only explained piecemeal across scattered comments, with no single definition). - Reconciled _check_open()'s docstring with the one call site (get_allocation_chunks_since) that doesn't hold the lock. - Closed coverage gaps found while re-reviewing the five fixes: use-after-close tests for import_external_tensor/import_peer_chunk/get_allocation_chunks_since (previously only allocate() was tested), a free-list-path rollback test for the failed-allocation fix (previously only the bump-pointer path was tested), a same-offset assertion proving the alignment fix actually reuses the freed block rather than coincidentally re-bumping into alignment, and two GPU-free driver-level tests pinning that get_minimum_granularity / get_recommended_granularity route distinct HIP flags. Co-authored-by: Cursor <cursoragent@cursor.com>
Abhishek-Varma
requested review from
BKP,
artulab,
drprajap,
efric,
mawad-amd,
neoblizz and
nirvedhmeshram
as code owners
November 6, 2026 09:14
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Fixes alignment, granularity, and rollback edge cases bugs in
VMemChunkedAllocator(refer #600)Fixes
Granularity reporting - distinguish minimum vs. recommended allocation
granularity.
BaseDriver.get_recommended_granularity()(a concrete default thatreturns the minimum) and query both minimum/recommended from the AMD
local and fabric drivers.
while reporting the true minimum via
get_stats()(
recommended_granularity/min_granularity).get_minimum_granularity()already returned the recommended value.
Alignment - honor a requested alignment larger than the granule.
allocate()computes an effective alignment (≥min_alignment, roundedto a power of two) and aligns the heap offset to it, keeping allocations
symmetric across ranks.
(size, alignment)- aBucket- so a block isonly reused for a request with matching alignment.
Use-after-close - add
_check_open()guards toallocate(),import_external_tensor(),import_peer_chunk(), andget_allocation_chunks_since(). Operations afterclose()now raiseRuntimeErrorinstead of silently corrupting allocator state.Failed-allocation rollback - build the tensor before committing the
bump pointer and bookkeeping. If tensor construction raises, a popped
free-list block is returned to its bucket and the bump pointer is left
untouched - no orphaned reservation, so a rank can always re-converge to
the symmetric layout.
Non-power-of-two heap sizes - drop the power-of-two
chunk_sizerequirement. VA reservation is rounded up to a whole number of chunks
arithmetically (
math.ceil) instead of via a bitmask, so arbitrarygranularity-aligned chunk sizes work for any heap size.
Testing
allocate()).block, rather than coincidentally landing on an aligned bump.
get_minimum_granularity()/get_recommended_granularity()routedistinct HIP flags.