[faresobeid]

Engines label the tokens they sample with the policy step, and the orchestrator logs per-token policy age.

• /update_weights takes weight_version and calls update_weight_version after loading. /load_lora_adapter takes a weight_version query parameter and does the same. The orchestrator sends the step on both. vLLM 0.32 then returns weight_versions spans on every /inference/v1/generate choice; PrimeRlServingTokens already passes them through.
• token_policy_ages turns the spans on a trace's assistant nodes into per-token ages against the batch's training version. The orchestrator logs off_policy/token/{mean,max} next to the existing episode-level off_policy/*.
• The boot step is labelled too. The startup sync (WeightWatcher.sync_startup → receive(v0 or the resume step) → /update_weights) runs before the dispatcher starts, so every train token carries a real step. Only Dynamo engines, whose updates bypass /update_weights, stay "default"; those tokens are not counted.
• Bumps deps/renderers to feat(client): pass through vLLM weight_versions spans renderers#195 and deps/verifiers to feat(v1): store engine weight-version spans on assistant nodes verifiers#2894 (merged), which carry the field. uv.lock follows verifiers' prime-sandboxes>=0.4.7.

Dynamo deployments keep the "default" label: their weight updates go through Dynamo's /collective_rpc, not our /update_weights. P/D: the router merges routed_experts by editing the JSON in place, so weight_versions passes through.

Tests: pytest tests/unit/orchestrator tests/unit/test_configs.py tests/unit/inference -m "not gpu", ruff.

GPU check (1×H200, Qwen3-0.6B, vLLM 0.32, this branch): a 4000-token generation paused mid-way, /update_weights with weight_version=5, resumed. It returned [{"version": "default", "start": 0, "end": 1390}, {"version": "5", "start": 1390, "end": 4000}]; the next request returned [{"version": "5", "start": 0, "end": 8}].

Upstream: vllm-project/vllm#59582 (merged, in 0.32) — the weight_versions spans this labels.

---

Note

Medium Risk
Extends the weight-update and LoRA reload admin path with mandatory weight_version; a mismatch or missing field could break updates, but behavior is additive labeling after successful loads.

Overview
Policy-step labeling on vLLM after each weight swap: /update_weights and /load_lora_adapter now receive the training step from the orchestrator and call vLLM’s update_weight_version, so generations return weight_versions spans (vLLM 0.32) on sampled tokens.

Finer off-policy metrics: new token_policy_ages / node_policy_versions turn those spans into per-token ages vs the batch’s training version; shipped batches log off_policy/token/mean and off_policy/token/max alongside existing episode-level off_policy/*. Tokens still labeled "default" (e.g. Dynamo) are excluded from token metrics.

Tests cover LoRA admin requests carrying weight_version and the age aggregation logic.

^{Reviewed by Cursor Bugbot for commit 41804d8. Bugbot is set up for automated code reviews on this repo. Configure here.}