Fuse rms_norm, mul, quantize_q8_1

[lnigam]

Overview

Fuse rms_norm+ mul+ qunatize_q8_1

Additional information

Tested on Qwen-3.6-35B-A3B-Q4_KM model, out of 81 RMS norms 41 are fused and 40 remained unfused due to incompatible MoE gate_input which requires F32 input only.

On 5090, it gives around 3-5% perf boost:
command: ./llama-bench -m ~/trust/Qwen3.6-35B-A3B-Q4_K_M.gguf -n 32 -ngl 500 -r 5 -fa 1 -d 0,100,512,4096 -p 0

Without change:

model	size	params	backend	ngl	fa	test	t/s
qwen35moe 35B.A3B Q4_K - Medium	19.82 GiB	34.66 B	CUDA	500	1	tg32	220.76 ± 8.60
qwen35moe 35B.A3B Q4_K - Medium	19.82 GiB	34.66 B	CUDA	500	1	tg32 @ d100	248.71 ± 11.13
qwen35moe 35B.A3B Q4_K - Medium	19.82 GiB	34.66 B	CUDA	500	1	tg32 @ d512	245.73 ± 10.62
qwen35moe 35B.A3B Q4_K - Medium	19.82 GiB	34.66 B	CUDA	500	1	tg32 @ d4096	242.65 ± 10.62

With-change:

model	size	params	backend	ngl	fa	test	t/s
qwen35moe 35B.A3B Q4_K - Medium	19.82 GiB	34.66 B	CUDA	500	1	tg32	232.91 ± 9.05
qwen35moe 35B.A3B Q4_K - Medium	19.82 GiB	34.66 B	CUDA	500	1	tg32 @ d100	259.02 ± 10.00
qwen35moe 35B.A3B Q4_K - Medium	19.82 GiB	34.66 B	CUDA	500	1	tg32 @ d512	255.50 ± 10.40
qwen35moe 35B.A3B Q4_K - Medium	19.82 GiB	34.66 B	CUDA	500	1	tg32 @ d4096	251.02 ± 9.78

Requirements

• I have read and agree with the contributing guidelines
• AI usage disclosure: yes, mostly for code review

[am17an]

Changing the node like this is going to cause issues, we should find another way