Skip to content

Map the DP's block MAP_NORESERVE, so a banded alignment is not refused for the whole matrix - #15

Merged
glennhickey merged 1 commit into
mainfrom
map-dp-block
Oct 29, 2026
Merged

glennhickey merged 1 commit into
mainfrom
map-dp-block

Conversation

@glennhickey

Copy link
Copy Markdown

The DP's block is sized for every row at full width, though a banded DP writes only the front of it. Linux, under its default overcommit policy, refuses an allocation larger than its memory and swap however little of it is touched, so pgrealign's banded alignments of long tandem-repeat alleles failed with posix_memalign fail on machines that could have run them: 62 kb against 70 kb reserves 128 GB and writes 5.5 GB.

A block of 64 MB or more is now mapped with MAP_NORESERVE (abpoa_matrix_alloc, abpoa_matrix_free in abpoa_simd.c), which Linux does not count against memory and swap; a smaller one, or one on a platform without MAP_NORESERVE, is allocated as before. Which one a block is follows from its size, so the struct and every other file are unchanged. A failed mapping exits as a failed posix_memalign did, with [abpoa_matrix_alloc] mmap fail! and the size.

🤖 Generated with Claude Code

…d for the whole matrix

The DP's block is sized for every row at full width, though a banded DP writes only the front of it.
Linux, under its default overcommit policy, refuses an allocation larger than its memory and swap
however little of it is touched, so pgrealign's banded alignments of long tandem-repeat alleles failed
with "posix_memalign fail" on machines that could have run them: 62 kb against 70 kb reserves 128 GB
and writes 5.5 GB. A block of 64 MB or more is now mapped with MAP_NORESERVE, which Linux does not count
against memory and swap; a smaller one, or one on a platform without it, is allocated as before, and
which one a block is follows from its size, so the struct and every other file are unchanged.

The rows written, and so the alignments and the memory they take, are the same: byte-identical output
and peak memory on 24 pgrealign alignments of 5 to 49 kb in every mode it uses (unbanded, banded,
seeded, several sequences), and on cactus's evolver primates with abPOA in BAR (MAF and halStats
byte-identical, 103 s both). tests/run_all.sh passes. The 62 kb pair now runs in 4 s and 5.5 GB.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@glennhickey
glennhickey merged commit 02628aa into main Oct 29, 2026
8 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant