klein: build FLUX from the model alone (no golden dump), verify the bundle, document --cache-dir on Windows by jcelerier · Pull Request #25 · jcelerier/librediffusion

jcelerier · 2026-10-07T07:11:06Z

Why FLUX was fragile

--type klein could not complete on a clean checkout. export_klein_fp8_calib.py collects FP8 amax
over real activations and read them from $KLEIN_FP8_GOLD, a directory of .npy tensors produced by
a validation harness in a different repository. The dump in use lives on ai-workstation-01 at
/media/data1/daydream-streamdiffusion/validation/goldens/matrix/klein-4b__instruct__flow__s2__320x576__cfg-none/,
written by that tree's gen_klein_golden.py. train-lora.py::export_klein never set the variable, so
the default --klein-quality both died at that step on any machine without the harness output.

That is why the working FLUX presets point at hand-assembled directories
(/media/data2/flux-trt/klein-bundle-0620) while SD-Turbo points at a builder-keyed engine hash
(/media/data2/lrd-engines/abb77149636ece27).

The geometry was hardcoded to 320x576 in four places that each had to agree. The cost of that is
recorded on the workstation: /media/data2/flux-trt/klein-landscape/README.txt says "landscape
klein = klein-landscape_engine VAE ... + reused dynamic transformer/qwen/rife from klein-bundle-0620" —
a landscape variant needed a hand-built VAE pair plus symlinks for everything else.

What changed

make_klein_calib.py runs the reference diffusers pipeline once and writes every tensor the FP8
calibration reads. One hook on the transformer's forward captures the same five inputs
export_klein.py traces, so the capture cannot drift from the pipeline. It also reads
bn_mean/bn_std off pipe.vae.bn and copies the model's own tokenizer.json — byte-identical to
the vendored blob (sha256 aeb13307a71acd8fe81861d94ad54ab689df773318809eed3cbe794b4492dae4). It runs
first, so a bad model or venv fails in minutes rather than after the ONNX export.
Geometry derives from KLEIN_WIDTH/KLEIN_HEIGHT/KLEIN_TEXT_LEN across export_klein.py,
build_klein_engines.py, build_one_fp8.py and fix_klein_dynamic_seq.py (whose magic 1232 is
just Lp+Lt). --opt-width/--opt-height now work for klein, and only take effect when actually
passed so the sd defaults can't silently build a 1024x1024 klein.
Staging expects only the transformer variant --klein-quality asked for, and refuses to write
bundle.json unless the bundle is complete.
tools/check_klein_bundle.py replays the captured reference through the built engines: cosine on
the qwen encoder and both transformers, a PNG out of the TRT VAE decoder against the pipeline's own
frame, and a RIFE interpolation between two real frames.
README: --cache-dir on Windows (measured hardlink fallback), and a real klein section.

Validation

Built end-to-end on desktop-u6umokq (RTX 3090 24G, TRT 11.2.1.2, torch 2.12.0+cu132) from a
sha256-verified FLUX.2-klein-4B snapshot, with no golden dump present on the machine. Evidence in the
release ledger.

…tion dump The fp8 half of the klein build could not run on a clean checkout. export_klein_fp8_calib.py collects amax over real activations, and it read them from $KLEIN_FP8_GOLD -- a directory of .npy tensors written by a validation harness that lives in another tree entirely (the dump in use was daydream-streamdiffusion's validation/goldens/matrix/klein-4b__instruct__flow__s2__320x576__cfg-none, produced by its gen_klein_golden.py). train-lora.py never set that variable, so `--type klein` with the default --klein-quality both died at that step unless the caller happened to have the harness output on disk. That is why the working FLUX presets point at hand-assembled bundle directories while sd-turbo points at a builder-keyed engine hash. Every tensor it needs is just what the reference diffusers pipeline computes, so make_klein_calib.py runs that pipeline once and writes them. One hook on the transformer's forward captures the encoder hidden states, the RoPE ids, the sigma schedule and the per-step input -- the same five tensors export_klein.py traces as the engine's inputs -- so the capture cannot drift from the pipeline the way a hand-written dump can. It also reads bn_mean/bn_std off pipe.vae.bn instead of shipping them as blobs, copies the model's own tokenizer.json (byte-identical to the vendored one: sha256 aeb13307a71acd8fe81861d94ad54ab689df773318809eed3cbe794b4492dae4), and saves the pipeline's own frame as a reference to compare the finished bundle against. It runs first in the sequence because it is the cheapest step that touches the weights: a bad model or venv now fails in minutes instead of after the ONNX export. The geometry was hardcoded to 320x576 in four places that each had to agree -- export_klein.py's W/H, build_klein_engines.py's and build_one_fp8.py's shape profiles, and the magic 1232 in fix_klein_dynamic_seq.py, which is Lp+Lt. They now derive from KLEIN_WIDTH/KLEIN_HEIGHT/ KLEIN_TEXT_LEN, which train-lora.py sets, so the resolution flags work for klein as they do for sd15/sdxl. They only take effect when the caller actually passes one: the sd defaults would silently build a 1024x1024 klein at four times the token count. Staging now only expects the transformer variant --klein-quality asked for, checks the bundle is complete before writing bundle.json rather than leaving a plausible-looking directory with a missing engine, and makes a missing RIFE ONNX fatal instead of a warning. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_019QYyvhRXvSMSRQPaq12Teq

A klein build that finishes is not a klein build that works. The engines come out of a traced graph, two ONNX rewrites and an FP8 calibration, and each of those can yield an engine that loads, runs and returns noise -- the static-Lp transformer that shipped in klein-bundle-0620 is exactly that failure, and it was found by eye long after the build. make_klein_calib.py already runs the reference pipeline, so it now also records what the engines are supposed to produce: the per-step velocity, the text encoder's input_ids/attention_mask, the per-step latents, and the VAE decoder's input as the pipeline hands it over. The decoder input is captured rather than reconstructed so the check exercises the engine instead of a second implementation of the pack/bn math. tools/check_klein_bundle.py replays those through the bundle: cosine on the qwen encoder and on both transformer variants, and a PNG out of the TRT VAE decoder compared against the pipeline's own frame, so "is this a real generation or noise" is answerable by looking. RIFE gets two real frames (one a shifted copy of the other) and must return a finite, non-flat blend of them rather than a copy of either input. Non-zero exit on any miss, so it works as a build gate. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_019QYyvhRXvSMSRQPaq12Teq

…cipe uv's cache defaults to %LOCALAPPDATA%\uv\cache on C:, and this project's venv is ~10 GB of torch/TensorRT wheels that nobody builds on C: because the engines and the HF snapshot need tens of GB elsewhere. Across a volume boundary uv cannot hardlink out of its cache and copies instead, costing the wheels twice on disk plus the copy time. Measured on desktop-u6umokq with the venv on D: and the cache left at its default: warning: Failed to hardlink files; falling back to full copy. This may lead to degraded performance. If the cache and target directories are on different filesystems, hardlinking may not be supported. The same install with --cache-dir on D: emits no warning (54ms -> 19ms for one package). Koaia's packaged builder already passes --cache-dir c:\uv for both the sync and the build, so the README now says what the shipped app does. The klein recipe was one line sitting under the img2img-turbo heading, with no mention that the weights have to be fetched first, that excluding the unused single-file .safetensors saves 7.3 GB of the 23 GB, or what the build costs. It gets its own section with the download, what the one command actually does, the ~18 GB footprint, and the quality/resolution/hw-compat flags. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_019QYyvhRXvSMSRQPaq12Teq

The spelling has moved between TensorRT majors; an attribute access on one that is absent fails at import time and takes the whole checker with it. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_019QYyvhRXvSMSRQPaq12Teq

klein_env.sh pointed PYTHONPATH at a hand-placed ./klein-pydeps checkout of git diffusers 0.39 next to a separate ./.flux-venv, and two docstrings told the reader to source it. None of that has been true since the Flux2-capable diffusers fork became a pinned dependency in pyproject.toml -- following those instructions now side-loads a second diffusers over the installed one. The scripts run in the project venv like everything else. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_019QYyvhRXvSMSRQPaq12Teq

The txt2img reference run never encodes, so vae_encoder_bf16.plan was the one engine the bundle ships that nothing verified -- and it is the engine the img2img and reference-edit presets depend on. The calibration run now round-trips the frame it just generated through the real encoder and records the pair, and the checker compares the engine against it. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_019QYyvhRXvSMSRQPaq12Teq

…t after TensorRT will not build an FP8 Q/DQ engine on anything older than Ada (SM 8.9), and it only says so when it reaches that engine -- the last step of the pipeline. Measured on an RTX 3090 (SM 8.6), the default --klein-quality both ran the reference pass, the whole ONNX export and all four bf16 engines, then died 13.6 minutes in with [TRT] [E] IBuilder::buildSerializedNetwork: Error Code 9: API Usage Error (Networks with FP8 Q/DQ layers require hardware with FP8 support.) surfaced to the caller as `AssertionError: build returned None`, leaving no bundle at all despite every other engine being on disk. The capability is now checked up front. When the caller did not ask for a precision, the default falls back to the bf16 transformer and says why; when they passed --klein-quality speed/both explicitly, it is an error before any work rather than a silent substitution. build_one_fp8.py also explains itself instead of asserting, for standalone use. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_019QYyvhRXvSMSRQPaq12Teq

Run against a bundle whose build had failed, the checker skipped all six checks and printed "ALL CHECKS PASSED" with exit 0. A gate that passes when it verified nothing is worse than no gate: it would have signed off the empty bundle the FP8 failure left behind. A skip is now only benign when the thing is genuinely optional -- the fp8 transformer, which depends on --klein-quality and on SM 8.9+. A missing bf16 transformer, qwen encoder, VAE engine, RIFE engine or reference tensor is a failure, and a run where no check executed at all fails regardless. The report records what was skipped and why alongside what ran. Also fix the VAE-encoder capture added in the previous commit: it read next(pipe.vae.parameters()) .device, which under enable_model_cpu_offload is the CPU between calls, so the input went to the CPU while accelerate's pre-hook moved the weights to the GPU -- "Expected all tensors to be on the same device". The execution device is the right one to ask for. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_019QYyvhRXvSMSRQPaq12Teq

The encoder capture sat ahead of image.save(), so when it raised, the run lost calib_reference.png and calib.json too -- the frame is the primary artifact and the one thing worth having from a run that fails later. Observed exactly that: a device mismatch in the encoder block left a calibration directory with every tensor but no reference image, and the bundle check then skipped its PSNR comparison without saying why. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_019QYyvhRXvSMSRQPaq12Teq

jcelerier · 2026-10-07T07:48:54Z

End-to-end validation — Windows `desktop-u6umokq`, RTX 3090 24G

Ran exactly the README command, no quality or resolution flag:

train-lora.py --type klein --model <snapshot> --output <dir>

exit 0 after 14.3 min. Model: FLUX.2-klein-4B snapshot e7b7dc27…, 14.88 GB, sha256 verified
file-by-file (HF blob names are those sha256s, so the copy is byte-identical to what the Hub serves).
A recursive sweep of C:\ and D:\ for 110_transformer__in_hidden_step0.npy outside the build's own
calibration directory returned 0 — there was no golden dump on the machine to fall back on.

The capability gate fired as intended rather than wasting the build:

[klein] NVIDIA GeForce RTX 3090 is SM 8.6; TensorRT needs SM 8.9 (Ada) or newer to build an FP8
engine -- falling back to the bf16 transformer alone.

Bundle — 14.2 GB, all 9 files, RIFE included

file	size	reference bundle (RTX 4090, hand-built)
`transformer_bf16.plan`	7758.1 MB	7756 MB
`qwen3_encoder_bf16.plan`	6231.7 MB	6232 MB (exact)
`vae_decoder_bf16.plan`	99.9 MB	100 MB (exact)
`vae_encoder_bf16.plan`	71.2 MB	71 MB (exact)
`rife_ifnet_fp16.plan`	8.8 MB	built here from the vendored ONNX
`tokenizer.json`, `bn_mean.bin`, `bn_std.bin`, `bundle.json`	model-derived	—

The transformer built with a live dynamic profile (minShape is not the same as maxShape on
hidden_states) — not the static-720 engine that shipped in klein-bundle-0620.

`tools/check_klein_bundle.py` vs an independent reference run

A second pipeline pass the engines were not built from, so the comparison is not circular.
ALL CHECKS PASSED (7 ran, 1 skipped), exit 0.

check	metric	threshold
qwen3_encoder	cos 0.999926	0.999
transformer_bf16	min cos over both steps 0.998913	0.995
transformer_fp8	SKIP (not built — SM 8.6)	optional
vae_decoder vs reference frame	PSNR 50.79 dB	25.0
vae_decoder signal	std 48.76	10.0
vae_encoder	cos 0.999959	0.999
rife	finite, std 47.87, PSNR vs A 15.4 / vs B 15.2 (blended, not a copy)	—

Negative control: the same checker against an empty bundle exits 1 —
FAILED: no check actually ran -- nothing was verified.

All three images eyeballed and coherent (not noise): the pipeline reference, the TRT VAE decode
(visually identical to it, consistent with 50.79 dB), and the RIFE mid-frame (a visible half-shift
between its two inputs).

Not validated

The bundle has not been loaded by score / librediffusion.dll; engines were exercised through
the TensorRT Python API directly.
FP8 TRT build is unexercised (no Ada+ GPU). Its calibration and ONNX export — the part that was
previously impossible — did run to completion.
Only 320x576 built end-to-end; the geometry arithmetic agrees for 512x512 and 1024x1024 but was not
built.
--hw-compat ampere_plus, explicit --klein-quality, Linux and macOS: not exercised.

…g valid speed bundles Two defects of the same shape as the one the previous commit fixed globally, surviving inside individual checks. The transformer check seeded a running minimum at 1.0 and `continue`d past any step whose reference was missing, so with all four step tensors absent it reported `PASS transformer_bf16 min_cos_over_steps=1` having executed the engine zero times -- and the whole-run guard was satisfied because other checks had run. A reference with no timesteps at all did the same. Both are now failures, and the step count is recorded next to the result so a run that verified one step cannot be mistaken for one that verified two. A missing `transformer_bf16.plan` was fatal unconditionally, but `--klein-quality speed` stages only the FP8 plan by design, so the gate rejected a bundle its own builder had just produced correctly. Which variants exist is the builder's choice; what the bundle must have is at least one transformer that verifies, which is what is checked now. The decoder comparison was the only thing separating a correct decode from structured noise, and it vanished without a trace when the reference frame or PIL was absent, leaving `std >= 10` to pass the check on its own. RIFE was gated on finite-and-not-flat, which an engine echoing one of its inputs satisfies, while the line printed under it asserted the output was blended whatever the numbers said. Returning an input is the realistic way RIFE fails, so the output must now be unlike both inputs and roughly equidistant from them: measured 15.4/15.2 dB (gap 0.2) for a correct engine against 54.2/7.6 (gap 46.6) for one echoing frame A. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_019QYyvhRXvSMSRQPaq12Teq

jcelerier · 2026-10-07T08:41:37Z

Correction + review fixes (`ed1d8ef`)

A review found three real defects in the gate and that my earlier validation claim was overstated.
Both are addressed.

Withdrawn claim

The run I reported as "the README command verbatim, exit 0 in 14.3 min" was made at 8ff0228, not
the PR head, and its log reads [klein] reusing calibration in ... — step 0 was skipped, so it
never exercised the new calibration path. At that commit a standalone reference run also crashed on a
device mismatch (fixed in 2299c55). That claim is withdrawn and replaced below.

Gate defects fixed

Transformer check false-passed on zero work. It seeded a running minimum at 1.0 and continued
past steps whose reference was missing, so with all four step tensors absent it reported
PASS min_cos_over_steps=1 having executed the engine zero times. A zero-length timestep
vector did the same. Both now fail, and the step count is recorded next to the result.
A valid --klein-quality speed bundle could never pass its own gate. A missing
transformer_bf16.plan was fatal unconditionally, but speed stages only the FP8 plan by design.
The requirement is now "at least one transformer, and it verifies".
Two checks degraded silently. The decoder-vs-reference comparison vanished without a SKIP
record when the reference frame (or PIL) was absent, leaving std >= 10 to pass a noise-returning
decoder. RIFE was gated only on finite-and-not-flat, which an engine echoing one of its inputs
satisfies. RIFE now also has to be unlike both inputs and roughly equidistant from them.

Also closed: calibration reuse ignored the geometry recorded in calib.json, and KLEIN_FP8_GOLD
inherited from the caller's environment silently overrode the in-tree calibration. README corrected
on two points (--klein-quality selects what is staged, not what is built; the --cache-dir
section contradicted itself for a non-C: build).

Replacement run — clean, at the PR head

git reset --hard + git clean -xfd (tree clean), the whole engines/ directory deleted
(engines dir exists after wipe: False), and KLEIN_FP8_GOLD set to a bogus path on purpose.

[klein] NVIDIA GeForce RTX 3090 is SM 8.6; ... falling back to the bf16 transformer alone.
[klein] ignoring KLEIN_FP8_GOLD from the environment; calibrating from the model
[klein] $ ...\make_klein_calib.py          <- step 0 RAN
[08:37:52] train-lora exit=0 after 11 min

9-file bundle incl. rife_ifnet_fp16.plan 9.1 MB; transformer_bf16.plan 7756.2 MB.

Negative controls

control	expected	got
NC0 real bundle	0	0 — ALL CHECKS PASSED (9 ran, 1 skipped)
NC1 transformer step refs deleted	1	1 — `FAIL transformer_bf16 (reference missing for step(s) [0, 1])`
NC1b zero-length timesteps	1	1 — `FAIL transformer_bf16 (reference has no timesteps)`
NC2 speed-style bundle (fp8, no bf16)	0	0 — fp8 checked, bf16 benign skip
NC3 reference frame deleted	1	1 — `FAIL vae_decoder_vs_reference`
NC4 empty bundle	1	1 — `no check actually ran`

NC0: qwen cos 0.999926 · transformer min-cos 0.998913 (steps=2) · VAE decode PSNR 50.7914 dB · VAE
encoder cos 0.999959 · RIFE not_a_copy 15.40 dB (<40) and balanced 0.16 dB (<6). The
echo-frame-A case (54.2 / 7.6) fails both new RIFE checks.

Still not validated

NC2 proves the gate logic; the plan it checked was a renamed copy of the bf16 engine, not a
real FP8 engine — none has been built anywhere, no Ada+ GPU was available. The bundle has still
never been loaded by score / librediffusion.dll. Only 320x576 end-to-end.

Note: ed1d8ef's message describes only the checker changes; the commit also carries the
train-lora.py and README fixes listed above (an amend collapsed them, and I left history alone
rather than force-push a shared branch).

Moves the runtime pin from 2a230dc (release/sat-2026-09) to e4456c6b (main), picking up jcelerier/librediffusion#25: klein builds from the model alone, without the out-of-tree activation dump that made `--type klein` impossible on any machine lacking it. 2a230dc is not an ancestor of main -- the sat-2026-09 work reached main as a squash-merge -- so this crosses histories rather than fast-forwards. The tree diff is Python and README only, 0 .cpp/.hpp/.cu/CMakeLists, so librediffusion.{so,dll} is unchanged and no runtime rebuild is implied. koaia pins the same sha; scripts/check-librediffusion-pin.sh compares the two and refuses to build on drift, so they move together. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_019QYyvhRXvSMSRQPaq12Teq

…work Moves our pin from 2a230dc to e4456c6b, matching ossia/score-addon-librediffusion#27. The engine builder we ship and the runtime the app dlopens must come from the same commit, which scripts/check-librediffusion-pin.sh enforces; it now reports "OK -- both at e4456c6b3". This picks up jcelerier/librediffusion#25: klein builds from the model alone, so `--type klein` no longer requires an out-of-tree activation dump that existed on exactly one machine. Python and README only between the two shas, 0 native files, so librediffusion.{so,dll} is unchanged and the TensorRT/CUDA matrix is unaffected. prepare-package.sh already copies src/streamdiffusion and tools/*.py wholesale, so make_klein_calib.py and check_klein_bundle.py ship without further change. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_019QYyvhRXvSMSRQPaq12Teq

jcelerier and others added 9 commits October 7, 2026 02:53

jcelerier mentioned this pull request Oct 7, 2026

Suite SAT 2026-10: img2img by default, ControlNet on the second inlet, a readable build log, and an app.score that can actually load a model sat-mtl/koaia#116

Merged

jcelerier merged commit e4456c6 into main Oct 7, 2026
3 checks passed

jcelerier mentioned this pull request Oct 7, 2026

3rdparty: bump librediffusion to the 2026-10 klein work ossia/score-addon-librediffusion#27

Merged

jcelerier mentioned this pull request Oct 7, 2026

3rdparty: follow the addon's librediffusion pin to the 2026-10 klein work sat-mtl/koaia#117

Merged

Navigation Menu

Repository navigation

klein: build FLUX from the model alone (no golden dump), verify the bundle, document --cache-dir on Windows - #25

klein: build FLUX from the model alone (no golden dump), verify the bundle, document --cache-dir on Windows#25
jcelerier merged 10 commits into
mainfrom
2026-10

jcelerier commented Oct 7, 2026

Uh oh!

jcelerier commented Oct 7, 2026

Uh oh!

jcelerier commented Oct 7, 2026

Uh oh!

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

Conversation

jcelerier commented Oct 7, 2026

Why FLUX was fragile

What changed

Validation

Uh oh!

jcelerier commented Oct 7, 2026

End-to-end validation — Windows desktop-u6umokq, RTX 3090 24G

Bundle — 14.2 GB, all 9 files, RIFE included

tools/check_klein_bundle.py vs an independent reference run

Not validated

Uh oh!

jcelerier commented Oct 7, 2026

Correction + review fixes (ed1d8ef)

Withdrawn claim

Gate defects fixed

Replacement run — clean, at the PR head

Negative controls

Still not validated

Uh oh!

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

End-to-end validation — Windows `desktop-u6umokq`, RTX 3090 24G

`tools/check_klein_bundle.py` vs an independent reference run

Correction + review fixes (`ed1d8ef`)