Eval bug: llama_model_load: error loading model: error loading model architecture: unknown model architecture: 'dflash'

[esmail-mkh]

Name and Version

version: 9405 (3764c5a53)
built with MSVC 19.44.35226.0 for Windows AMD64

Operating systems

Windows

GGML backends

CUDA

Hardware

RTX 3070 8GB + 32GB Ram

Models

Qwen 3.6 35B A3B IQ4NL

Problem description & steps to reproduce

when i want to use dflash model there is an error:

llama_model_load: error loading model: error loading model architecture: unknown model architecture: 'dflash'

how can i fix it?

First Bad Commit

No response

Relevant log output

print_info: file format = GGUF V3 (latest)
print_info: file type = Q8_0
print_info: file size = 480.40 MiB (8.50 BPW)
llama_model_load: error loading model: error loading model architecture: unknown model architecture: 'dflash'
llama_model_load_from_file_impl: failed to load model
srv load_model: failed to load draft model, 'M:\Qwen\Qwen3.6 35B A3B IQ4NL\Qwen3.6-35B-A3B-DFlash-Q8_0.gguf'
srv operator (): operator (): cleaning up before exit...
main: exiting due to model loading error

[Anbeeld]

Can't find the source for your draft model, https://huggingface.co/starskyzheng/Qwen3.6-35B-DFlash-GGUF uses dflash-draft same as 27B drafters I used, could you please provide a link to it?

[Anbeeld]

Tried https://huggingface.co/unsloth/Qwen3.6-35B-A3B-GGUF (IQ4) + https://huggingface.co/starskyzheng/Qwen3.6-35B-DFlash-GGUF (Q8) and can confirm it works, and decently fast. Still, I'm interested in the source of your drafter so I could confirm that some authors use dflash as architecture.

[esmail-mkh]

Tried https://huggingface.co/unsloth/Qwen3.6-35B-A3B-GGUF (IQ4) + https://huggingface.co/starskyzheng/Qwen3.6-35B-DFlash-GGUF (Q8) and can confirm it works, and decently fast. Still, I'm interested in the source of your drafter so I could confirm that some authors use dflash as architecture.

I tested this draft and it worked, there is no difference with original version for me

I get 25t/s from original and get 22t/s with draft

[Anbeeld]

1. What draft you tried initially?
2. What kind of prompts are you making?

[esmail-mkh]

1. What draft you tried initially?
2. What kind of prompts are you making?

this is my run command, i use Q8 Draft, and about prompt i tried all kind of prompt

llama-server.exe -m "M:\Apps\LMStudio_models\Qwen\Qwen 3.6 35B A3B IQ4 NL Uncensored\Qwen3.6-35B-A3B-Uncensored-HauhauCS-Aggressive-IQ4_NL.gguf" -c 128000 --temp 0.6 --top-p 0.95 --top-k 20 --min-p 0.0 --presence-penalty 1.2 --repeat-penalty 1.0 --fit on -fa on --jinja --cache-type-k turbo4 --cache-type-v turbo3_tcq --n-cpu-moe 35 -ub 2048 -np 1 --chat-template-file "qwen3.6/chat_template.jinja" --chat-template-kwargs "{"preserve_thinking":true}" --spec-type dflash --spec-draft-model "M:\Apps\LMStudio_models\Qwen\Qwen3.6 35B A3B IQ4NL\dflash-draft-3.6-q8_0.gguf" --spec-dflash-cross-ctx 1024 --spec-draft-ngl all

[Anbeeld]

Yes, I do see that current DFlash doesn't seem to help with 35B. For v0.1.0 the target was 27B, I'll try to follow up with 35B benchmarking and fine tuning in the near future. This might me architectural limitation of MoE vs dense, or it might be improper configuration. We'll see.

[esmail-mkh]

Yes, I do see that current DFlash doesn't seem to help with 35B. For v0.1.0 the target was 27B, I'll try to follow up with 35B benchmarking and fine tuning in the near future. This might me architectural limitation of MoE vs dense, or it might be improper configuration. We'll see.

Thank you very much, I hope it works better