Commit 53b3323
fix(deltanet): restore L2 norm (removal causes output collapse) + decay fix kept
Karpathy loop results on Qwen3.5-4B short prompts:
Loop 1: decay formula aligned with llama.cpp → still fails (0/5)
Loop 2: L2 norm removed → WORSE (doc QA also breaks)
→ L2 norm restored (REQUIRED for Qwen3.5)
The decay formula fix (sk before decay) is kept as it's
mathematically correct per llama.cpp reference.
Remaining suspect: Q scaling timing or state shape/layout mismatch.
The reference analysis found llama.cpp uses [S_v, S_v] square state
while quant.cpp uses [dk, dv] rectangular — this needs investigation.
Refs #95
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>1 parent d26ca5e commit 53b3323
1 file changed
+2
-0
lines changed| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
13890 | 13890 | | |
13891 | 13891 | | |
13892 | 13892 | | |
| 13893 | + | |
| 13894 | + | |
13893 | 13895 | | |
13894 | 13896 | | |
13895 | 13897 | | |
| |||
0 commit comments