Repository navigation
[2/3] Add request sampling controls to all pinned Drama engines - #27
Conversation
Restore the "answers as upstream does without flags" rule for the full-model patches and state the per-request temperature, top_p, top_k and seed fields from this PR (0013/0014, CUDA 0002, Metal 0004/0005) as the request-level exception: they need no launch flag, and requests without them are unchanged.
|
Thanks! The validation and the seed handling look right to me, and requests without the new fields keep exactly the old seeds and parameters. One non-blocking ask for TensorFold #6 (and so CUDA patch 0002): after this change |
|
My CUDA checks for #27 pass: seeds repeat byte for byte (including 2^53+1 and 2^64-1), out-of-range From my side this is ready. Please mark it ready for review whenever you're happy, and I'll merge it right after #26. The small README and patch-subject touch-ups for #28 I mentioned earlier I'll add once this one is in. |
Description and motivation
Part 2 of 3. Depends on #26; merge that first, then this PR, then #28. The incremental recipe change is 37a6832...cb20bda.
Expose per-request
temperature,top_p,top_kand exact unsigned 64-bitseedin all three pinned engines without changing their pins or default settings. Add full-model patches 0013/0014, Metal NVFP4 patches 0004/0005, and CUDA NVFP4 patch 0002, each with exact checksum coverage. Source companions: modern #1277 and legacy CUDA #6.Validation happens before engine submission. Tests inspect the actual sampler fields sent through whole/streamed requests, reference rebuilding, CFG and redraws. Explicit seeds retain all 64 bits; redraws add their take ordinal with wrapping arithmetic. Legacy long-form applies that policy separately to each segment. Documentation states the existing JSON-f64 underflow boundary and native sampler differences.
The public Sogni-AI/drama3-mlx API is the behavioral reference. Native Zig changes were implemented independently.
Testing
scripts/build-engine.shreapplies the complete selected series from each clean pin: full-model 66/66 build steps and Metal NVFP4 73/73 build steps passed on macOS arm64 with Zig 0.17.0. Native server and selected voice tools compile. Applied trees exactly equal reviewed source commits1c31b4d2and2af18fba.b1f4b9b4; tree equals reviewed22939c2a. Exact exported-tree CPU suite: 371/375 passed, four skipped.test-pull-hostaborts with ISCONN, andtf-flashnext-runfails on SharedEvent/Event. Both reproduce on the untouched base; this is not a passing whole-upstream build claim.During final CPU replays, two unchanged recall tests failed while the volume had about 3.3 GiB free, below their existing 4 GiB admission floor. Removing only task-created compiler caches restored 5.043 GiB; the identical commands passed with no source, environment or admission-guard change. Original failure receipts were retained.
Performance or quality claim
None. Equal settings do not promise equal audio across native backends.
Checklist