Commit ba8a615
fix(qwen35): suppress <think> token — Qwen3.5-4B short prompts now work (#95)
Root cause: NOT a DeltaNet implementation bug. Qwen3.5 defaults to
thinking mode (<think>...</think>), consuming all max_tokens budget
on reasoning before the actual answer. "What is 2+2?" generated
"<think>\n\n2+2=4\n\n</think>\n\n4" — the "4" was at token ~15,
beyond max_tokens=8.
Three fixes in tq_generate:
1. Suppress <think> logit to -1e30 before sampling (prevents entry)
2. Strip leading whitespace tokens (catches residual \n\n)
3. Skipped tokens don't count toward max_tokens budget
Results:
Before: "What is 2+2?" → "The answer to **" (FAIL)
After: "What is 2+2?" → "4" (PASS)
Document QA: still works (no regression)
Closes #95
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>1 parent 53b3323 commit ba8a615
1 file changed
+33
-1
lines changed| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
16175 | 16175 | | |
16176 | 16176 | | |
16177 | 16177 | | |
| 16178 | + | |
| 16179 | + | |
| 16180 | + | |
| 16181 | + | |
| 16182 | + | |
| 16183 | + | |
| 16184 | + | |
| 16185 | + | |
| 16186 | + | |
16178 | 16187 | | |
16179 | 16188 | | |
16180 | 16189 | | |
| |||
16191 | 16200 | | |
16192 | 16201 | | |
16193 | 16202 | | |
| 16203 | + | |
16194 | 16204 | | |
16195 | 16205 | | |
16196 | 16206 | | |
| |||
16286 | 16296 | | |
16287 | 16297 | | |
16288 | 16298 | | |
| 16299 | + | |
| 16300 | + | |
| 16301 | + | |
| 16302 | + | |
| 16303 | + | |
| 16304 | + | |
| 16305 | + | |
| 16306 | + | |
| 16307 | + | |
| 16308 | + | |
| 16309 | + | |
| 16310 | + | |
| 16311 | + | |
16289 | 16312 | | |
16290 | 16313 | | |
16291 | 16314 | | |
| |||
16307 | 16330 | | |
16308 | 16331 | | |
16309 | 16332 | | |
16310 | | - | |
| 16333 | + | |
| 16334 | + | |
| 16335 | + | |
| 16336 | + | |
| 16337 | + | |
16311 | 16338 | | |
16312 | 16339 | | |
16313 | 16340 | | |
| |||
16325 | 16352 | | |
16326 | 16353 | | |
16327 | 16354 | | |
| 16355 | + | |
| 16356 | + | |
| 16357 | + | |
| 16358 | + | |
| 16359 | + | |
16328 | 16360 | | |
16329 | 16361 | | |
16330 | 16362 | | |
| |||
0 commit comments