lfm3: serve LFM3-1.7B NVFP4 on CUDA, weight-only (stacked on #1513, #1492 and #1503) by jschmied · Pull Request #1508 · ashhart/TensorFold

jschmied · 2026-10-27T22:45:40Z

Part of #1491 (CUDA side). Stacked on BlivionIaG's LFM3 admission #1513 (head b3d9d6a4e, feature/lfm3-admission). His two commits are cherry-picked unchanged with his authorship: e003af71b (= 8192e5437) and 5e8e2f84d (= b3d9d6a4e). Below them are #1492 (LFM2.5 on CUDA) and #1503 (its pool pages and prompt cache). The ten commits after 5e8e2f84d are new here. It stays a draft until those land, then I'll rebase onto main. There is no root build-lfm3.zig.

What

LFM3-1.7B text served natively on CUDA from LiquidAI/LFM3-1.7B-NVFP4 (ModelOpt NVFP4, revision b30025bb). It is weight-only: the FFN's FP4 weights are used as stored, and every activation stays bf16. The export's static FP4 input scales are read, because the admission requires them, but not applied. The load says so in its log, and the loader's comment at cuda_weights.zig:118 says so too. MLX affine and FP8 come next on the same family; VL after that.

Admission (d7735968f, on top of lfm3: admit official MLX text and VL checkpoints #1513): the ModelOpt NVFP4 scheme in the shared lfm3/config.zig / weights.zig, with the export's exact exclusion list. Only the FFN (w1, w3, w2) is FP4; conv, attention, gate, embedding and head are bf16. The MLX layouts are unchanged.
Metal refuses the export by name (f8097cbbb): config.servedOn(config, .metal) returns Lfm3ModeloptNvfp4NotOnMetal for modelopt_nvfp4, with a test. Metal's LFM3 open in lfm3: serve the MLX text decoder on Metal #1514 calls it.
Kernels:
- 4a61a437f: the 7-tap instances of lfm2_moe: serve LFM2.5-24B-A2B on CUDA (plain serve, shared kernels) #1492's liquid_conv.cuh (tf_liquid_conv_7 / tf_liquid_commit_7), under the contract with controls.
- 554a30843: prefill_attention's template at head dim 128 with 4 query heads a KV head.
Family (7fd391ec2):
- FP4 FFN on TF's NVFP4 linears;
- q|k|v|gate as one bf16 lane weight, and a bf16 copy of the tied embedding as the head;
- 6 full-attention layers in pool pages, with partial RoPE and Nemotron's head-128 chunk/merge for decode;
- 5 sliding layers (window 256) as a per-slot ring on the shared window-attention kernels;
- the per-head gate (act_rows.headGate) and the shared norms.
One Liquid backend (48f23c96b): zig/src/families/liquid/ holds the lane backend, the kept-state walk and the native glue (open, admission, pool, prompt-cache hooks), generic over the family's model and forward, as liquid_conv.cuh is over taps.
- LFM2.5 and LFM3 each keep only their config, weights, forward, model and refusals. Their cuda_backend.zig files are 17 lines each.
- The commit is +1,048 / −1,486.
- A model names its fixed regions (LFM2.5: the conv states; LFM3: the conv states and each sliding layer's K and V rings) and its paged layers.
- Kept states live in pages allocated once inside the prompt-cache budget (lfm2_moe CUDA: attention pages from the shared KV pool, and the prompt cache (stacked on #1492) #1503's fix, now shared).
- Marks inside a prompt chunk are tapped where the family allows it (LFM2.5). LFM3 still ends a chunk at a mark, because the rest of the chunk would overwrite its sliding rings.
Contract (2c863784a): pattn_kernel<128, 8, 4, 8> against contract_attn.zig's float64 bound over shuffled pages, with a control (values from the neighbouring 8 dims).
Reference (942eab46b): tools/lfm3/ref64.py and its receipt REF64.md.
Docs (290975190): docs/recipes/lfm3-cuda.md, with the command, the revision, the weight-only section, the receipt and the licence.

Licence

The weights are under Liquid AI's LFM Open License v1.0 (license: other, lfm1.0; the text is the checkpoint's LICENSE), as the MLX exports are.

Use, changes and redistribution are allowed.
Commercial use is licensed only to an organisation under US$10 million in annual revenue.
Non-profit research and other non-commercial use are allowed regardless.

Verification (GB10, sm_121, real weights)

float64 reference (CPU, NVFP4 dequantized exactly; it agrees 829/829 with the official modeling_lfm3.py run in float32):
- greedy agrees on 825 of 829 teacher-forced prompt rows and 127 of 128 reply rows;
- with the engine's bf16 rounding points it is 827/829 and 126/128;
- REF64.md lists all 5 misses with both tokens and both margins (float64 margins 0.003–0.153).
Served LFM3, every token hash equal to the previous head's:
- drafted == plain 8/8, and concurrent == solo 8/8;
- prompts of 300–2,960 tokens across the window, 8/8;
- resumed == fresh over 8 chats × 4 turns (24/32 hits), equal to cache off and to fixed pages (TENSORFOLD_LFM3_KV_PAGES=0);
- a 0.1 GiB cache with 9 evictions matches;
- after every drain the pool reads 520/520 free.
Served LFM2.5 on this head (the shared backend): exact, turns and restart hashes are equal to lfm2_moe CUDA: attention pages from the shared KV pool, and the prompt cache (stacked on #1492) #1503's.
Machine code: every existing function is unchanged on sm_80/86/89/120/121 under nvcc 13.0 and 13.2: prefill_attention 40 functions (+5) and contract_controls 265 (+15).
Contract: the 7-tap conv and commit and pattn_kernel<128> pass gates 1–4. In the filtered runs (paged, liquid) all 13 controls go red, theirs among them. Limit records for the Liquid images pass on nvcc 13.0 and 13.2; the only failures are main's two stale qwen35 records.
lean_check: 0 new against lfm2_moe CUDA: attention pages from the shared KV pool, and the prompt cache (stacked on #1492) #1503's head. zig fmt --check zig and zig build test pass.

Speed (GB10)


decode, 1 / 4 / 8 streams	112.8 / 420.0 / 767.4 tok/s
cold TTFT, 2k / 8k / 32k	0.150 / 0.615 / 2.84 s
cold chat turn, 10,146 tokens, keeping two states (3 alternating rounds × 6)	0.807 s (previous head, per-keep allocation: 0.811 s)
warm turn, 10.25k tokens with 10.14k cached	0.049 s

The same cold turn with the cache off takes 0.774 s. The 31 ms keep cost is LFM3's two marks still cutting the prompt pass; taps that also save the rings would remove it. LFM2.5 on this head runs its cold turn in 1.218 s, against 1.216 s on #1503 and 1.211 s on #1492.

Only the cold-turn rows are new this round. Decode, the cold TTFTs and the warm turn are from the previous head.

Settled in review

The default context stays 32,768, as LFM2's does.
Fusing the head gate into the attention epilogue comes later, in its own PR with an A/B.

ashhart · 2026-10-27T23:56:46Z

Thanks @jschmied. A float64 reference that agrees with the official modelling code at every position is the evidence we want. Admitting only the export's exact ignore list keeps the scheme honest.

Before this leaves draft:

It stacks on an old copy of the admission. cb2d77003 still adds the root build-lfm3.zig, which lfm3: add native Metal vision support above the text decoder #1502 has since folded into build.zig. Please rebase onto the admission as it lands, and onto main once lfm2_moe: serve LFM2.5-24B-A2B on CUDA (plain serve, shared kernels) #1492 has.
zig/src/families/lfm3/cuda_backend.zig is lfm2_moe CUDA: attention pages from the shared KV pool, and the prompt cache (stacked on #1492) #1503's 426 lines with the names changed; cuda_snap.zig and cuda_native.zig are mostly the same as lfm2_moe CUDA: attention pages from the shared KV pool, and the prompt cache (stacked on #1492) #1503's too. Please share one lane backend, and the state walk, between the two Liquid families, generic over the model and the forward. liquid_conv.cuh already shares the conv that way.
pattn_kernel<128, 8, 4, 8> has a limit record (lfm3_cuda_records.zig:11) but no gate. Please gate it against contract_attn.zig's float64 reference, with a planted control, as we've asked on lfm2_moe: serve LFM2.5-24B-A2B on CUDA (plain serve, shared kernels) #1492 for head 64.
Each kept state allocates a device buffer and frees it behind a stream wait (cuda_snap.zig:61, :75-76), as in lfm2_moe CUDA: attention pages from the shared KV pool, and the prompt cache (stacked on #1492) #1503. The same fix applies.
Please commit the reference, and list each miss with its tokens and both margins.
Please label the mode weight-only: the export's static FP4 input scales are read but not applied (cuda_weights.zig:118).
Please state the weights' licence.

To your questions:

Keep 32,768 as the default, as LFM2 does, and raise it once 64k and longer contexts are measured.
Fuse the head gate later, in its own PR, with an A/B.
Yes, one admission for both backends. Whichever of this and lfm3: add native Metal vision support above the text decoder #1502 lands second should make Metal refuse modelopt_nvfp4 by name, with a test.

Then we'll run the contract and its limits. We'll serve the NVFP4 export on a GB10: drafted against plain, concurrent against solo, prompts across the sliding window, and resumed against fresh.

BlivionIaG · 2026-10-28T08:37:53Z

@jschmied Could you rebase the CUDA follow-up onto the updated LFM3 admission/Metal work, preserving both authorships, so the shared admission and build files have a common base?

The Metal chain is now split into admission #1513, text #1514 and vision #1502. The admission branch is BlivionIaG:feature/lfm3-admission; the combined Metal branch is BlivionIaG:feature/lfm3-vl-support (currently d05f599a3, including c12411bdd). Please coordinate the shared admission changes against #1513 and retain your #1492 dependency, then rebase onto upstream main as those foundations land, as requested above. The old root build-lfm3.zig should not be reintroduced.

The Metal chain remains draft while numerical and performance qualification continues. This request is to reconcile the overlapping admission/build changes; it does not claim the chain is ready to land.

The shared-window chunk and merge bodies move to paged_attn_segs.cuh, included twice by nemotron_attention.cu: Nemotron's head-128 kernels keep their names and SASS, and head 64 adds tf_paged_attn_chunk_segs_64 and tf_paged_attn_merge_segs_64. prefill_attention.cu instantiates pattn_kernel<64, 8, 4, 8> over paged caches. sassid on sm_80/86/89/120/121: 0 functions changed, 15 added.

lfm2_moe.cu holds what the shared operators do not: the short conv over each stream's state with every window row's snapshot, the commit of an accepted row's conv state, and the router (fp32 weights, top-4 of sigmoid + bias, ties to the lower id, a NaN ranked as -inf). Each has a planted control. Launch records cover these kernels and the head-64 paged attention on nvcc 13.0, with one 13.2 override.

The MLX 4-bit checkpoint (embedding at 4, 6 or 8 bits) on the shared affine-4 projections and experts, with prompt chunks of 1024 rows, one-row decode and copy-drafted windows over --parallel streams. The ten attention layers keep their keys and values in 512-position pages, and each stream has its own conv state. There is no drafter and no prompt cache yet.

…ount The gated short conv over each stream's state and the conv-state commit move out of lfm2_moe.cuh into liquid_conv.cuh, templated on the tap count, so LFM3 (7 taps) can take the same bodies as LFM2.5 (3). lfm2_moe.cu keeps the router and instantiates the 3-tap pair as tf_liquid_conv_3 and tf_liquid_commit_3; the controls become tf_ctl_liquid_conv_3 and tf_ctl_liquid_commit_3. cuobjdump -sass on sm_80/86/89/120/121, nvcc 13.0 and 13.2: the moved bodies are identical under their new names, every other function unchanged.

… against float64 tf_paged_attn_chunk_segs_64 with tf_paged_attn_merge_segs_64 and pattn_kernel<64, 8, 4, 8> run over shuffled 512-key pages, decode rows over up to eight streams a launch, and are held head by head to contract_attn's float64 bound. Two controls in contract_controls.cu read the values from the neighbouring lane's dims (decode) and from the neighbouring 8 dims (prompt rows). To build them from the same bodies, nemotron_attention.cu's page and tile helpers move to paged_attn.cuh and prefill_attention.cu's to prefill_attention.cuh; every existing function's SASS is unchanged on sm_80/86/89/120/121 under nvcc 13.0 and 13.2.

… of every top-1 miss ref64.py dequantizes the MLX 4-bit weights exactly and runs the forward in float64; --bf16 adds the engine's bf16 storage points. It checks the CUDA engine's greedy picks on teacher-forced prompt rows and on reply rows; REF64.md lists each miss with the prompt, the position, both tokens and both margins.

The recipe gets a CUDA section (build, serve, revision, GB10 receipt, licence), the README row names one NVIDIA GPU beside the Mac, and the contract's coverage lists LFM2.5's CUDA kernels.

…m leasing The kernels already read keys and values through a slot's page table, so the pool changes only what the table holds and how large the arena is; no kernel changes. A page is 512 positions of the ten attention layers' keys and values (10 MiB, 20 KiB a token); the conv states stay a fixed part a slot (240 KiB). - Loaded.layout with engine_paged, as Dory's: the plan sets a window of pages aside for the first stream; each stream leases its prompt, its reply and one verify window at its prompt pass and gives them back when it ends. A request that cannot lease waits (LaneMemory); fits is the host's admission. - Model.attach makes the arena and the tables and conv states of the slots the plan names; forward refuses rows past a slot's pages. The idle server logs the pool's census. - TENSORFOLD_LFM2_KV_PAGES=0 keeps every stream's own window of pages, made at open. Served on the GB10 at --parallel 8, 16k: drafted == plain and concurrent == solo, also with 8 long prompts on a 52-page pool where the last waits for pages; the replies equal the fixed pages' ones.

…the pages' rows through its position A state at `at` is the slot's conv states and its keys and values at positions [0, at) of the ten attention layers: one copy a run of consecutive whole pages, then the last page's rows a layer's keys and values. It is copied behind the chunk that ends at the mark and restored into the new stream's leased pages before its next chunk, so a later turn's pass starts at its history. Lookahead 0, as the other headless families. States live in 2 MiB snapshot pages allocated once at startup inside the prompt-cache budget (`snap_ready`), as GLM's: a keep or a restore leases or reads pages and allocates nothing, and a dropped state's pages go back to the pool at once (every copy that reads or writes them is queued on the backend's one stream). The store counts held and spare pages through `snap_charged` / `snap_spare` / `snap_reuses`.

…dds no pass A chat mark inside a prompt chunk ended the chunk there, and every extra prompt pass reads the weights once more: a cold 10k chat turn, whose two marks sit in its last chunk, ran 12 passes instead of 10. Now up to four marks inside a chunk are tapped instead: the conv kernel runs once more on each mark's last row alone, writing the same y and that row's conv states into the model's tap buffer, and the keep copies those states with the K/V rows before the mark. The pass goes on to its grid line; a fifth mark still ends it. The kernel is unchanged.

…rompt cache

Prepare text and VL 4/6/8-bit checkpoints with revision-pinned header inspection and native metadata validation. Inference remains unregistered pending operator and model qualification. Refs ashhart#1491

…KV head prefill_attention.cu instantiates pattn_kernel<128, 8, 4, 8>: Nemotron's <128, 8, 8, 8> needs eight query heads a KV head, and LFM3's full-attention layers have four (16 over 4). Its launch record is measured on nvcc 13.0 and 13.2 from every SM's image (206 registers on sm_80/86/89, 231 on sm_120/121, no static shared memory). sassid on sm_80/86/89/120/121, both toolkits: 0 functions changed, 5 added.

LiquidAI/LFM3-1.7B-NVFP4 stores only the FFN as NVFP4: w1, w3 and w2 as e2m1 codes in groups of 16 with e4m3 block scales, an fp32 tensor scale and a static input scale. Every mixer projection, the embedding and the tied head are bf16, and the depthwise conv kernel is [channels, 1, taps]. - config.zig reads `quantization_config` with quant_method "modelopt" as Scheme.modelopt_nvfp4: quant_algo NVFP4, one Linear group of static FP4 g16 weights and inputs, and an ignore list that is exactly the head, the embedding and every layer's mixer. Anything else is UnsupportedLfm3Quantization. MLX configs parse as before (Scheme.mlx_affine), so a decoder that reads one scheme can refuse the other by name. - weights.zig's schema takes that layout for the bare namespace. tf-lfm3-check passes the published header (b30025bb): 502 text tensors, 1,829,689,408 bytes. The MLX-4bit header still passes (594 tensors, 973,008,384 bytes). Host tests 26/26.

…tract lfm3.cu instantiates liquid_conv<7,0> and liquid_commit<7,0>; contract_controls.cu adds their <7,1> controls. contract_lfm2_moe.zig's Conv and Commit take the tap count; contract_lfm3.zig checks the served 7-tap shapes. Records are identical on nvcc 13.0 and 13.2. contract_limits.zig now lists the lfm2_moe and lfm3 images.

Conv layers keep a 6-row state per slot; full attention runs on kv_pool pages with partial RoPE (32 of 64 pairs); sliding attention runs on a 1280-row per-slot ring through the shared window kernels at window 256. FFN w1|w3 is one FP4 lane weight (shared weight_scale_2), w2 FP4; q|k|v|gate, o, conv and the tied head are bf16. Adds --parallel lanes, drafted rows, and a copy-based prompt cache (conv states, rings, pages [0, at)).

…and LFM3 on CUDA zig/src/families/liquid/ holds what the two Liquid families shared as copies: the lane backend (cuda_lanes.zig), the kept states in the prompt cache's preallocated pages and the walk over a slot's state (cuda_snap.zig), and the native glue: open, admission, the pool and the prompt-cache hooks (cuda_native.zig). Each is generic over the family's model and forward, as liquid_conv.cuh is over the conv's taps. A model names its fixed regions (LFM2.5: the conv states; LFM3: the conv states and every sliding layer's rings) and its paged layers; each family keeps its config, weights, forward, refusals and model. lfm2_moe's backend, snap and native are now thin instances, and lfm3's no longer copy them.

…4, with a control LFM3's full-attention prompt rows (16 query heads over 4 of 128) join the head-64 paged attention's gates, held head by head to contract_attn's float64 bound over shuffled 512-key pages; tf_ctl_pattn_128 reads its values from the neighbouring 8 dims.

config.servedOn(config, backend) refuses modelopt_nvfp4 on Metal with Lfm3ModeloptNvfp4NotOnMetal; CUDA serves it, and both serve the MLX layouts. Metal's LFM3 open (ashhart#1514) calls it.

The export's static FP4 input scales are read, as the admission requires, and not applied: activations stay bf16. The load logs it and the loader's comment says so.

…top-1 miss ref64.py dequantizes the NVFP4 weights exactly and runs the forward in float64; --bf16 adds the engine's bf16 storage points. REF64.md lists each miss on teacher-forced prompt rows and reply rows with both tokens and both margins.

…ts' licence

jschmied · 2026-10-28T11:40:24Z

Thanks @ashhart. All seven are done; the head is 290975190.

Rebase: onto lfm3: admit official MLX text and VL checkpoints #1513's head b3d9d6a4e, at BlivionIaG's request. His two commits are unchanged with his authorship (e003af71b, 5e8e2f84d), and our NVFP4 admission is our own commit on top (d7735968f). There's no root build-lfm3.zig. lfm2_moe: serve LFM2.5-24B-A2B on CUDA (plain serve, shared kernels) #1492 and lfm2_moe CUDA: attention pages from the shared KV pool, and the prompt cache (stacked on #1492) #1503 (both rebased) stay below. I'll rebase onto main as those land.
One backend: zig/src/families/liquid/ holds the lane backend, the kept-state walk and the native glue, generic over the family's model and forward. LFM2.5's and LFM3's cuda_backend.zig are 17 lines each, and their cuda_snap.zig and cuda_native.zig only name the model, the checks and their tests. A model names its fixed regions (LFM2.5: conv states; LFM3: conv states and the sliding K/V rings) and its paged layers. The commit is +1,048 / −1,486. LFM2.5 served on this head gives the same token hashes as lfm2_moe CUDA: attention pages from the shared KV pool, and the prompt cache (stacked on #1492) #1503.
Gate: pattn_kernel<128, 8, 4, 8> runs against contract_attn.zig's float64 bound over shuffled 512-key pages. Its served shapes are 256 rows with no history, 64 after 3,000 keys and 100 after 500, with planted NaN/inf. It passes gates 1–4, and its control, which reads values from the neighbouring 8 dims, goes red at 121× the bound. prefill_attention's existing 35 functions and contract_controls' 250 are unchanged on 5 SMs under nvcc 13.0 and 13.2.
Pages: lfm2_moe CUDA: attention pages from the shared KV pool, and the prompt cache (stacked on #1492) #1503's fix, now shared. Kept states are in pages allocated once inside the prompt-cache budget, so a keep or restore allocates nothing and a drop only returns the lease. The cold 10k chat turn is 0.807 s against 0.811 s before.
- LFM3 still ends a chunk at each chat mark, because the rest of the chunk would overwrite its sliding rings. That costs 31 ms against cache off on that turn.
- LFM2.5 taps its marks instead (lfm2_moe CUDA: attention pages from the shared KV pool, and the prompt cache (stacked on #1492) #1503). Doing that for LFM3 needs a ring snapshot at the tap, so I'd leave it for a follow-up.
Reference: tools/lfm3/ref64.py with REF64.md. 825/829 prompt rows and 127/128 reply rows agree in float64, and 827/829 and 126/128 with bf16 points. All 5 misses are listed with both tokens and both margins.
Weight-only: said in the load log, at cuda_weights.zig:118, in the recipe and in the PR title. The static FP4 input scales are read, because the admission requires them, but not applied.
Licence: LFM Open License v1.0, in the recipe's Licence section and the PR body.

Metal refusal: #1513 is still open, so this PR adds it. config.servedOn(config, .metal) refuses modelopt_nvfp4 with Lfm3ModeloptNvfp4NotOnMetal, and there's a test. #1514's Metal open needs to call it.

The served checks were rerun on this head, and every token hash is equal to the previous head's:

drafted == plain and concurrent == solo, 8/8;
prompts across the sliding window, 8/8;
resumed == fresh, 24/32 hits, equal to cache off and to fixed pages;
evictions and a restart.

jschmied · 2026-10-28T11:40:25Z

@BlivionIaG Done: #1508 now stacks on #1513's head b3d9d6a4e (feature/lfm3-admission), not on the combined branch.

Your two commits are unchanged and keep your authorship. They're e003af71b (your 8192e5437) and 5e8e2f84d (your b3d9d6a4e), with only new SHAs from the base underneath.
Our NVFP4 admission change is our own commit on top, d7735968f. f8097cbbb adds config.servedOn(config, backend), which refuses modelopt_nvfp4 on Metal by name, with a test. lfm3: serve the MLX text decoder on Metal #1514's Metal open would call it.
There's no root build-lfm3.zig; the admission builds through your build.zig wiring.
lfm2_moe: serve LFM2.5-24B-A2B on CUDA (plain serve, shared kernels) #1492 and lfm2_moe CUDA: attention pages from the shared KV pool, and the prompt cache (stacked on #1492) #1503 stay below it.

This was referenced Oct 27, 2026

lfm2_moe: serve LFM2.5-24B-A2B on CUDA (plain serve, shared kernels) #1492

Draft

lfm3: add native Metal vision support above the text decoder #1502

Draft

BlivionIaG mentioned this pull request Oct 28, 2026

lfm3: admit official MLX text and VL checkpoints #1513

Draft

ashhart mentioned this pull request Oct 28, 2026

Add native Metal support for LFM3-1.7B and LFM3-VL-2B MLX 4/6/8-bit checkpoints #1491

Open

jschmied and others added 24 commits October 28, 2026 10:42

kernel contract: the limits check reads lfm2_moe's fatbin

22eba94

docs: LFM2.5-24B-A2B on CUDA, its receipt and the weights' licence

7390fb3

The recipe gets a CUDA section (build, serve, revision, GB10 receipt, licence), the README row names one NVIDIA GPU beside the Mac, and the contract's coverage lists LFM2.5's CUDA kernels.

docs: LFM2.5 on CUDA leases pool pages and keeps chat states in the p…

0506d9c

…rompt cache

feat(lfm3): add strict MLX checkpoint admission checks

e003af7

Prepare text and VL 4/6/8-bit checkpoints with revision-pinned header inspection and native metadata validation. Inference remains unregistered pending operator and model qualification. Refs ashhart#1491

build(lfm3): isolate admission in the standard build graph

5e8e2f8

lfm3: Metal refuses the ModelOpt NVFP4 export by name

f8097cb

config.servedOn(config, backend) refuses modelopt_nvfp4 on Metal with Lfm3ModeloptNvfp4NotOnMetal; CUDA serves it, and both serve the MLX layouts. Metal's LFM3 open (ashhart#1514) calls it.

lfm3: say at load that NVFP4 on CUDA is weight-only

e8ec4f7

The export's static FP4 input scales are read, as the admission requires, and not applied: activations stay bf16. The load logs it and the loader's comment says so.

docs: LFM3-1.7B NVFP4 on CUDA, weight-only, its receipt and the weigh…

2909751

…ts' licence

BobClawblaw mentioned this pull request Oct 28, 2026

MOSS-Image: the denoiser on CUDA (GB10, RTX 5090) #1528

Open

jschmied force-pushed the lfm3-cuda-nvfp4 branch from 08b338c to 2909751 Compare October 28, 2026 11:38

jschmied changed the title ~~lfm3: serve LFM3-1.7B NVFP4 on CUDA, weight-only (stacked on #1492 and the LFM3 admission)~~ lfm3: serve LFM3-1.7B NVFP4 on CUDA, weight-only (stacked on #1513, #1492 and #1503) Oct 28, 2026

Navigation Menu

Repository navigation

lfm3: serve LFM3-1.7B NVFP4 on CUDA, weight-only (stacked on #1513, #1492 and #1503) - #1508

lfm3: serve LFM3-1.7B NVFP4 on CUDA, weight-only (stacked on #1513, #1492 and #1503)#1508
jschmied wants to merge 24 commits into
ashhart:mainfrom
jschmied:lfm3-cuda-nvfp4

jschmied commented Oct 27, 2026 •

edited

Loading

Uh oh!

ashhart commented Oct 27, 2026

Uh oh!

BlivionIaG commented Oct 28, 2026

Uh oh!

jschmied commented Oct 28, 2026

Uh oh!

jschmied commented Oct 28, 2026

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Conversation

jschmied commented Oct 27, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What

Licence

Verification (GB10, sm_121, real weights)

Speed (GB10)

Settled in review

Uh oh!

ashhart commented Oct 27, 2026

Uh oh!

BlivionIaG commented Oct 28, 2026

Uh oh!

jschmied commented Oct 28, 2026

Uh oh!

jschmied commented Oct 28, 2026

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

jschmied commented Oct 27, 2026 •

edited

Loading