Skip to content

feat(client): pass through vLLM weight_versions spans - #195

Merged
faresobeid merged 1 commit into
mainfrom
feat/weight-versions
Oct 19, 2026
Merged

faresobeid merged 1 commit into
mainfrom
feat/weight-versions

Conversation

@faresobeid

@faresobeid faresobeid commented Oct 18, 2026 •

Copy link
Copy Markdown
Contributor

Pass vLLM's weight_versions choice field through generate(). vLLM >= 0.32 returns, per choice, [{version, start, end}] spans over completion-token indexes: which engine weights sampled each token. None when the engine labels none.

No-op on vLLM 0.31: the field is absent, so the value is None. It can land before prime-rl's vLLM 0.32 bump (PrimeIntellect-ai/prime-rl#4140). Consumers: PrimeIntellect-ai/verifiers#2894 (stores the spans on the assistant node), then PrimeIntellect-ai/prime-rl#4141 (engines label the policy step; per-token policy age).

Test: pytest tests/test_client.py (32 passed).


Note

Low Risk
Additive response field pass-through with no parsing or behavior change when the engine omits the key.

Overview
generate() now surfaces vLLM’s per-choice weight_versions field on its return dict, copied from the engine response via choice.get("weight_versions").

Docs describe it as vLLM ≥ 0.32’s list of {version, start, end} spans over completion-token indexes (which weight snapshot produced each token), or None when the engine omits or labels none—so older vLLM stays a no-op.

Reviewed by Cursor Bugbot for commit 0526944. Bugbot is set up for automated code reviews on this repo. Configure here.

Note

Add weight_versions to generate async client result

The async generate client function in client.py now returns the selected response choice's vLLM weight_versions span value, or None when absent. The field's structure and None case are documented on the function.

Macroscope summarized 0526944.

@macroscopeapp

macroscopeapp Bot commented Oct 19, 2026

Copy link
Copy Markdown

Approvability

Verdict: Approved at 0526944

Macroscope's review found this PR approvable — This is a narrowly scoped, backwards-compatible addition to the existing generate() response that passes through optional vLLM weight_versions metadata without changing requests or existing behavior. Older vLLM responses remain a no-op with None.

Notes:

  • Macroscope's correctness review did not run, so approvability was decided on eligibility alone.

You can add or adjust custom eligibility rules. Learn more.

@faresobeid
faresobeid merged commit 7c0e9e9 into main Oct 19, 2026
11 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants