Skip to content

feat(cpu-alloc): run the lvstore poller group on the nvmf poller cores - #1452

Merged
wmousa merged 1 commit into
mainfrom
feat/lvol-poller-on-poller-cores
Oct 14, 2026
Merged

wmousa merged 1 commit into
mainfrom
feat/lvol-poller-on-poller-cores

Conversation

@wmousa

@wmousa wmousa commented Oct 8, 2026

Copy link
Copy Markdown
Contributor

Summary

The lvol poller group was pinned to a single core: the JC singleton's below 32 vCPU, a dedicated one above. One busy core then throttled every lvstore and transfer-hub bdev at once. The 2026-09-29 test build (26f57dd72, MERGE_LVOL_POLLER_WITH_POLLER_CORES, since reverted) checked that by handing the nvmf pollers and the lvol poller group one shared core set at mask-build time.

This makes it the allocation itself, no flag:

  • calculate_core_allocations() derives lvol_poller_core from poller_cpu_cores in every tier (utils._lvol_poller_cores), so lvol_poller_mask, still the single source of truth that add-node and restart send to bdev_lvol_create_poller_group, is the multi-core poller mask.
  • No core is reserved for it. Below 32 vCPU the layout is exactly what it was. At 32 and above the core the old layout dedicated to the lvol poller returns to the distrib/alceml/poller pool, so the base roles take 3 cores at every size; the 24-core distrib step therefore lands at 37 vCPU instead of 38.
  • The JC singleton core is the fallback only when a layout has no poller core at all (a 2-vCPU host, or a device count that clips the remainder to nothing).
  • Persisted nodes keep their stored masks (restart does not re-derive the role allocation), so the placement applies to nodes added or configured after the upgrade.

Tests

  • tests/unit/test_lvol_poller_on_poller_cores.py replaces test_lvol_poller_jc_colocation.py: every tier shares the poller set, no core reserved at ≥32, the multi-core mask, the JC fallback, the tiny tiers.
  • test_distrib_priority_over_alceml.py: the 5-vCPU layout, the exclusivity check and the distrib-count table updated for the 3-core base.
  • ruff and mypy clean on the changed modules; the unit suite's remaining failures are the pre-existing ones on main (test_spdk_proxy_unit, test_lblk_persistent_names), identical with the change stashed.

Not yet exercised on a cluster; the vm12 cluster is mid-test on the current images.

🤖 Generated with Claude Code

@github-actions github-actions Bot added the target/main Targets the development branch label Oct 8, 2026
@wmousa
wmousa force-pushed the feat/lvol-poller-on-poller-cores branch from 0c7746c to 605af4f Compare October 8, 2026 15:25
The lvol poller group was pinned to one core: the JC singleton's below
32 vCPU, a dedicated one above. One busy core then throttled every
lvstore and transfer-hub bdev at once, which is what the 2026-09-29
test build (26f57dd, MERGE_LVOL_POLLER_WITH_POLLER_CORES) set out to
check by handing both the nvmf pollers and the lvol poller group one
shared core set at mask-build time.

Make that the allocation itself instead of a flag applied after the
fact: calculate_core_allocations() derives lvol_poller_core from
poller_cpu_cores in every tier (utils._lvol_poller_cores), so
lvol_poller_mask - still the single source of truth add_node and
restart send to bdev_lvol_create_poller_group - is the multi-core
poller mask. No core is reserved for it: below 32 vCPU the layout is
exactly what it was, at 32 and above the core the old layout dedicated
to the lvol poller returns to the distrib/alceml/poller pool, so the
base roles take 3 cores at every size (the 24-core distrib step now
lands at 37 vCPU rather than 38). The JC singleton core remains the
fallback only when a layout ends up with no poller core at all (a
2-vCPU host, or a device count that clips the remainder to nothing).

Persisted nodes keep their stored masks: restart does not re-derive
the role allocation, so the new placement applies to nodes added or
configured after the upgrade.

Replace the jc-colocation tests with ones for the new rule and update
the tiny-tier and exclusivity assertions that encoded the old one.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
@wmousa
wmousa force-pushed the feat/lvol-poller-on-poller-cores branch from 605af4f to 97421ae Compare October 14, 2026 21:32
@wmousa
wmousa merged commit 66b2f8e into main Oct 14, 2026
18 checks passed
@wmousa
wmousa deleted the feat/lvol-poller-on-poller-cores branch October 14, 2026 22:51
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

target/main Targets the development branch

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant