Skip to content

[3/3] Preserve dialogue speaker roles across all pinned Drama engines - #28

Draft
krunkosaurus wants to merge 5 commits into
plotarmordev:mainfrom
krunkosaurus:contrib/dialogue-speakers
Draft

krunkosaurus wants to merge 5 commits into
plotarmordev:mainfrom
krunkosaurus:contrib/dialogue-speakers

Conversation

@krunkosaurus

@krunkosaurus krunkosaurus commented Oct 25, 2026 •

Copy link
Copy Markdown
Contributor

Description and motivation

Part 3 of 3. Merge #26, then #27, then this PR. The incremental change is cb20bda...0b2e000.

Accept canonical <|speaker:0|> through <|speaker:15|> tags in speech input and reference transcripts. Untagged input remains speaker 0; malformed/structural tokens still fail validation. Full-model/Metal catalogs and the Python writer retain their four-reference cap; legacy CUDA retains its existing body/file/prompt limits.

Append full-model patch 0015, Metal NVFP4 patch 0006 and CUDA NVFP4 patch 0003, and update the Python catalog writer, examples, indexes and checksums. Source companions: modern #1278 and legacy #7. Legacy continuation carries the active speaker across chunks, paragraph cuts, history eviction and redraws. Patch 0001 is required to keep speaker markers intact at hard cuts; the combined regression also exercises explicit sampling and an exact seed above 2^53.

The public Sogni-AI/drama3-mlx prompt tests are the behavioral reference. This independently extends TensorFold's native prompt builder; no Fish-derived Python implementation is copied.

Testing

  • Model-free CLI admission test: 1/1 passed; prefill-comparison parser tests: 5/5 passed. Stream-check self-test passed on byte-string fixtures (not a live audio stream).
  • Final Python recipe suite: 48 tests in 8.010 seconds, 15 skipped, zero failures. Tests cover all sixteen IDs, exact transcript/code round trips, malformed/sign/leading-zero/non-ASCII tags and the unchanged reference cap.
  • Full-model composed CPU suite: 577/577 passed. Metal NVFP4: 748/751 passed, three skipped. Legacy CUDA: 379/383 passed, four skipped. Tests use synthetic codes/scripted PCM; these are not model/audio-quality runs.
  • All 24 patch digests verify with exact unique manifest coverage. Whole series apply from their unchanged pins; resulting source trees exactly equal reviewed commits c8463349 (full), a6de32a3 (Metal), and 9e782dc9 (CUDA).
  • Final scripts/build-engine.sh runs on those complete series passed: 66/66 full-model steps and 73/73 Metal NVFP4 steps, including the native server and selected voice tools (macOS arm64, Zig 0.17.0).
  • Final modern combined source at 60eb5d21: 784/787 CPU/host tests passed, three skipped, 9/9 build steps. Independent agent review verified the merge retains both feature implementations and test modules.
  • Modern speaker source before stacking: 777/780 focused CPU/host tests passed, three skipped. Four acceptance tests fail before the implementation. Equal-length reference-role changes invalidate the prefix cache; request-only role changes retain a matching reference prefix.
  • Independent adversarial agent implementation, composition and final packaging reviews passed. Review found and fixed the legacy hard-cut prerequisite, reference-limit documentation and a stale qualification label. No new native lean findings; applied-source and non-patch recipe whitespace checks pass.
  • Broader modern source checks encountered two unchanged macOS base failures: test-pull-host aborts with ISCONN, and tf-flashnext-run fails on SharedEvent/Event. Both reproduce on the untouched base.
  • Real CUDA host compilation is unavailable on this Mac (the unchanged base uses O.DIRECT). Model/device inference, audio identity, voice-similarity listening and qualification gates remain unrun; the launcher memory admission floor was not met and was not overridden. This PR remains a draft.

During final CPU replays, two unchanged recall tests failed while the volume had about 3.3 GiB free, below their existing 4 GiB admission floor. Removing only task-created compiler caches restored 5.043 GiB; the identical commands passed with no source, environment or admission-guard change. Original failure receipts were retained.

Performance or quality claim

None. Speaker labels do not guarantee distinct or faithfully cloned voices.

Checklist

  • Ordered appended patches, checksums, indexes and source links updated.
  • Exact source-tree equality for every complete patch series.
  • Actual recipe build script compiles the full-model and Metal series from clean pins.
  • Native/Python regressions and independent adversarial agent review.
  • No weights, private voices, recordings or private machine paths published.
  • CUDA build/device and real-model/audio qualification.

@plotarmordev

Copy link
Copy Markdown
Owner

Thanks for carrying the speaker through long-form continuations so carefully. The tests around hard cuts and redraws are great. Two small things: CUDA patch 0003's subject starts with "Drama speech:", while the series uses server:. I'll adjust that in a small commit on your branch, along with the README line that lists speaker tags as a request-level exception. Also, the drama3-mlx chunker caps a chunk at three speakers; is it intentional that the legacy CUDA split has no such cap? Before merging I want to run a listening check of multi-speaker output on my GB10, since this moves away from the reference's speaker-0-only rule. I'll report back here.

plotarmordev added a commit that referenced this pull request Oct 25, 2026
gpu-checks: GPU check and D1-D4 diagnostic harness for PRs #26-#28
@plotarmordev

Copy link
Copy Markdown
Owner

My CUDA checks for #28 mostly pass: tags 1 and 15 are accepted, untagged requests are unchanged, and when a long-form reply reaches its second segment the continuation carries the right speaker (5 of 8 seeds reached it, all carried correctly).

One thing I'd like your view on before I merge: on the full model (--weights full), a tagged two-speaker line with a catalog voice ran to the frame cap in 3 of 5 seeds (finish=length, 47.55 s for "<|speaker:1|> Is the train here? <|speaker:15|> It arrives at seven fifteen."). The same text untagged stopped normally in 5 of 5, and NVFP4 with the same tags stopped normally too. Do you see a reason the tags could keep the full model from emitting its stop, or is that something you've seen on Metal as well? I'm happy to run any variant you'd like on CUDA, since you can't test it there.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants