perf: fuse unquantized Qwen3-Next GDN projections

[T4t4KAU]

Summary

• Fuse QKV/Z and B/A projections in csrc/models/qwen3_next/qwen3_next_gated_deltanet.* for unquantized GDN, reducing input projection GEMMs from four to two.
• Add dense merged projection support in csrc/layers/linear/fused_linear.* and preserve checkpoint parameter names and TP shard metadata. Refresh parameter aliases after weight pre-transposition using logical checkpoint views in csrc/layers/quantization/none_quantization.cpp.
• Keep the existing SiLU followed by multiplication. Quantized GDN retains the main branch's projection construction, execution order, and weight-processing behavior.

Motivation

Small-batch GDN decode pays launch overhead for separate projections of the same hidden states. Combining their dense weights reduces layer latency by 12.40%–18.86% in the measured BF16 decode cases. Short prefill is 0.77%–0.93% slower; long prefill is approximately unchanged. Detailed measurements and methodology are below. No linked issue.

Type of Change

• feat — new feature / new model
• fix — bug fix
• perf — performance improvement (no behavioral change)
• refactor — code restructuring without behavior change
• test — adding or fixing tests only
• docs — documentation only
• build / ci — build system or CI configuration
• chore — tooling, formatting, or other non-code changes
• Breaking change

Test Results of Involved Models on Supported Platforms (Please attach screenshots)

Platform / model	Result	Coverage / limitation
NVIDIA RTX 5090 ×4, synthetic complete Qwen3-Next GDN layer	PASS: 12 configurations, 48 rank reports	Real NCCL TP=4; FP16/BF16/FP32; pre-transpose on/off; decode B=4 and ragged prefill lengths 43/44/44; output and conv/SSM state comparisons; initial eager/Graph parity; eight subsequent eager decode steps with carried state.
NVIDIA RTX 5090, FP16 GDN boundary	PASS	Z=-12, norm scale 128, pre-transpose on, single-device TP=4/rank=3 slice, decode B=4, CUDA Graph; nonzero output, zero output/state difference.
NVIDIA RTX 5090 ×2, Qwen3-8B real checkpoint	PASS	TP=2, paged attention, CUDA Graph, deterministic 16-token generation. Pre-transpose on/off produced identical tokens; checks shared parameter-view behavior.
Full Qwen3-Next checkpoint	Not run	Checkpoint unavailable. Layer tests do not establish whole-model accuracy or throughput.
Quantized GDN	Not validated	This PR applies fusion only to QuantScheme::NONE; no quantized performance claim.
Other supported hardware	Not run	No non-NVIDIA hardware available in the test environment; platform CI remains pending.

Tests were run noninteractively and recorded as text/JSON, so terminal screenshots were not captured. Results and selected output are included below. The four-GPU tests used NCCL 2.27.3+cuda12.9. The standalone harnesses and raw logs are retained outside the source tree; they are not part of this PR.

Dtype	Max initial output relative L2	Max continuation output relative L2	Max state relative L2	Threshold
float16	0.00054433226	0.00047725392	0.00030541772	0.003
bfloat16	0	0	0	0.01
float32	1.1535943e-05	0.00011468848	9.1628313e-07	0.002

Recorded validation output

Fresh-directory release build (compiler cache enabled):

[100%]: build ok, spent 13.311s

FP16 boundary result (single device exercising TP=4, rank=3):

{"boundary":true,"dtype":"float16","max_abs":0.0,"max_reference_abs":0.0006895065307617188,"mode":"graph","norm_scale":128.0,"output_ok":true,"packing_control":false,"phase":"decode","pre_transpose":true,"rank":3,"reference_norm":0.016309970758792116,"relative_l2":0.0,"state_relative_l2":{"conv":0.0,"ssm":0.0},"tokens":4,"tolerance":0.003,"tp":4}

Qwen3-8B completion token IDs with pre-transpose disabled, then enabled:

{"pre_transpose":false,"token_ids":[151667,198,32313,11,279,1196,4588,752,311,3270,279,3409,330,14990,3263,6771]}
{"pre_transpose":true,"token_ids":[151667,198,32313,11,279,1196,4588,752,311,3270,279,3409,330,14990,3263,6771]}

The named example/service scripts were not run: examples/test_infer.py, examples/bench.py, test/bench/test_benchmark.py, and python/infinilm/server/inference_server.py + scripts/test_perf.py. A full Qwen3-Next checkpoint is unavailable. The standalone GDN comparison and Qwen3-8B LLM API smoke test are the available validation, not substitutes for whole-model accuracy and service coverage.

Benchmark / Performance Impact

One NVIDIA RTX 5090, CUDA 12.8, BF16, TP=1, pre-transpose disabled. The standalone C++ harness runs a complete GDN layer with hidden size 2048, 16 key heads, 32 value heads, key/value head dimensions 128, and convolution kernel size 4. Decode inputs are [1, B, 2048] with B independent one-token requests; prefill inputs are [1, T, 2048] with one request. Baseline GDN source is identical to main at 8657a94, with only its namespace renamed so both implementations can run in one process. Weights and inputs use a fixed seed (20260911).

Each case warms up both variants for one second, alternates their order over nine rounds, and reports median CUDA-event latency. Repetitions per round are 500/495 for decode, 31 for 128-token prefill, and 5 for 2048-token prefill. Timed calls read fixed initial recurrent state and write separate final slots; this measures layer latency, not end-to-end generation throughput. Separate continuation tests validate carried state. All eight BF16 performance cases have zero output difference against the baseline.

Phase	Tokens	Execution	Main (µs)	PR (µs)	Latency change
decode	1	eager	78.07	63.34	-18.86%
decode	1	graph	51.20	43.08	-15.87%
decode	16	eager	196.26	168.70	-14.04%
decode	16	graph	183.24	160.52	-12.40%
prefill	128	eager	3140.11	3169.45	+0.93%
prefill	128	graph	3123.04	3147.23	+0.77%
prefill	2048	eager	49253.25	49108.58	-0.29%
prefill	2048	graph	49227.58	49093.62	-0.27%

Negative latency change means faster. These are complete-layer measurements, not projection-only or whole-model throughput numbers. The small prefill regression is retained in the results.

Notes for Reviewers

• MergedColumnParallelLinear stores Q/K/V/Z in one allocation. The named parameters must continue to alias the active GEMM weight after pre-transposition; NoneQuantization::split_params exposes logical [OC, IC] checkpoint views over the packed [IC, OC] storage.
• Q/K/V/Z and B/A outputs are strided views. Review the consumer stride assumptions and the TP shard metadata alongside the loading lifecycle.
• The two fused modules register their split parameters on the GDN parent, so the parent explicitly invokes their post-load processing. They are dense layers without quantized runtime buffers.
• The PR is intentionally limited to unquantized projections. The activation formula, public configuration, and checkpoint keys are unchanged.
• Five C++ files changed. There are no added test files, build targets, dependencies, or public Python API changes.
• The branch contains one commit (c6522ec) directly on current main (8657a94).

CI / ChatOps

Local formatting passed with the CI-pinned clang-format 21.1.8. A fresh-directory release build passed on NVIDIA (compiler cache enabled).

For head commit c6522ec:

• CI / Check Format: action_required; no passing result yet.
• Ruff: action_required; no passing result yet.
• Hardware CI: not run. The ci job is gated on workflow_dispatch in .github/workflows/ci_test.yml.

Maintainer action is needed to allow the PR workflows to run and arrange hardware CI. The corresponding checklist items remain open. No reviewer has been tagged and no ChatOps command has been posted.

---

Checklist

Title, Branch, and Commits

• PR title follows Conventional Commits: perf: fuse unquantized Qwen3-Next GDN projections.
• Branch name follows <type>/xxx-yyyy-zzzz: perf/fuse-gdn-projections.
• Each commit message follows Conventional Commits.
• Small PR is a single squashable commit.
• No stray merge commits from main; the branch is based directly on current main.
• No fixup! / squash! / wip commits remain.
• N/A: Existing PR/branch/commit using the legacy issue format — new branch uses Conventional Commits.

Scope and Design

• Changes are minimal and related to the stated motivation.
• No dead code, commented-out blocks, debug prints, or unowned TODOs were introduced.
• No unrelated formatting churn obscures the diff; existing initializer indentation is normalized by the required formatter in the touched file.
• N/A: Public API changes — no public configuration or Python API change.

General Code Hygiene (applies to all languages)

• Comments explain the non-obvious parameter alias invariant.
• Modified files end with a single trailing newline.
• No trailing whitespace, tab/space mixing, or stray BOMs.
• Identifiers in new error messages are wrapped in backticks.
• All added comments and error messages are in English.
• Comments use complete sentences; errors follow the C++/LLVM diagnostic convention.

C++ Specific (if C++ files changed)

• Changes follow the Google C++ Style Guide and the project's .clang-format.
• Error messages follow the LLVM diagnostic convention.
• Constructor initializer order matches member declaration order.
• Smart pointers and existing allocators are used; no raw new/delete added.
• Changed files pass scripts/format.py --ref origin/main --check.
• No changes/reference to csrc/models/llama_legacy/.

Python Specific (if Python files changed)

• N/A: No Python files changed; PEP 8, Python comments/docstrings, Python formatting, and python/infinilm/auto_config.py checks do not apply.

Testing

• For untested platforms, reasons are given above; reviewers with hardware access have not been tagged.
• Single request test (examples/test_infer.py), or specify the reason for skipping — skipped; full Qwen3-Next checkpoint unavailable. Qwen3-8B LLM API smoke passed.
• Offline performance test (examples/bench.py), or specify the reason for skipping — skipped for the same checkpoint limitation; layer performance is reported above.
• Sanity test (test/bench/test_benchmark.py), or specify the reason for skipping — skipped for the same checkpoint limitation.
• Service test (python/infinilm/server/inference_server.py + scripts/test_perf.py), or specify the reason for skipping — skipped for the same checkpoint limitation.

Build, CI, and Tooling

• Project builds from a fresh directory on an affected NVIDIA platform (compiler cache enabled).
• Hardware CI has been triggered manually or /retest requested — pending.

Documentation

• N/A: No user behavior, build flags, or developer workflow changes requiring README/CONTRIBUTING updates; the parameter alias invariant is documented inline.
• N/A: No user-visible breaking change.

Security and Safety

• No secrets, tokens, internal URLs, customer data, or personal hardware identifiers are included in the commit.
• N/A: No third-party code added.
• No raw pointer arithmetic or uninitialized reads introduced; split construction validates sizes, shard divisibility, and unique nonempty names.