Repository navigation
feat(client): pass through vLLM weight_versions spans - #195
Merged
Merged
Conversation
faresobeid
marked this pull request as ready for review
October 19, 2026 10:05
hallerite
approved these changes
Oct 19, 2026
ApprovabilityVerdict: Approved at Macroscope's review found this PR approvable — This is a narrowly scoped, backwards-compatible addition to the existing Notes:
You can add or adjust custom eligibility rules. Learn more. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Pass vLLM's
weight_versionschoice field throughgenerate(). vLLM >= 0.32 returns, per choice,[{version, start, end}]spans over completion-token indexes: which engine weights sampled each token.Nonewhen the engine labels none.No-op on vLLM 0.31: the field is absent, so the value is
None. It can land before prime-rl's vLLM 0.32 bump (PrimeIntellect-ai/prime-rl#4140). Consumers: PrimeIntellect-ai/verifiers#2894 (stores the spans on the assistant node), then PrimeIntellect-ai/prime-rl#4141 (engines label the policy step; per-token policy age).Test:
pytest tests/test_client.py(32 passed).Note
Low Risk
Additive response field pass-through with no parsing or behavior change when the engine omits the key.
Overview
generate()now surfaces vLLM’s per-choiceweight_versionsfield on its return dict, copied from the engine response viachoice.get("weight_versions").Docs describe it as vLLM ≥ 0.32’s list of
{version, start, end}spans over completion-token indexes (which weight snapshot produced each token), orNonewhen the engine omits or labels none—so older vLLM stays a no-op.Reviewed by Cursor Bugbot for commit 0526944. Bugbot is set up for automated code reviews on this repo. Configure here.
Note
Add
weight_versionstogenerateasync client resultThe async
generateclient function in client.py now returns the selected response choice's vLLMweight_versionsspan value, orNonewhen absent. The field's structure andNonecase are documented on the function.Macroscope summarized 0526944.