Bad audio quality with Llama.cpp

[acoracle]

Hi, I have tried to generate audio with Llama.CPP backend on CPU only, the generated sound is intelligible but sounds very metallic, very far from the quality you can get from the Torch based examples. Changing the level of GGUF quantization doesn't seem to make much difference. Is this normal? Is there any way to improve the quality?

[cms42]

Thank you for reporting this issue. The metallic sound you're experiencing is not typical, and we suspect it might be related to the tokenizer implementation. In audio generation models, the tokenizer plays a critical role in mapping text to the correct discrete tokens, and any discrepancy could lead to degraded audio quality.

Could you please clarify whether you are using ONNX Runtime on CPU for the tokenizer part? Also, to help us diagnose the issue, could you provide detailed environment information, including:

• llama.cpp version (commit hash or release tag)
• ONNX Runtime version (if applicable)
• ONNX execution provider (e.g., CPUExecutionProvider)
• Hardware (CPU model, RAM)
• Operating system (Windows/Linux/macOS and version)

If possible, you can use a tool like python -c "import onnxruntime; onnxruntime.print_debug_info()" or a similar script to gather the relevant ONNX Runtime details. This information will help us narrow down whether the problem is related to the tokenizer, the inference backend, or the environment configuration.

Looking forward to your response!

( As a workaround, you may try llama.cpp backbone + torch cpu tokenzier, if possible)

[acoracle]

Thanks, please find here the requested information, yes I am using onnxruntime for CPUs.

• Llama.cpp hash is ec947d2b16218580dfad3f2f8bd589190955efaf (10/3)
• onnxruntime 1.24.3
• providers ['AzureExecutionProvider', 'CPUExecutionProvider'], we are using CPUExecutionProvider
  -CPU model name : AMD EPYC 9J45 128-Core Processor, VM with 16 cores, and 64 GB of DDR5 RAM
• Ubuntu 22.04, Sys 3.14.3 | packaged by Anaconda, Inc. | (main, Feb 24 2026, 22:51:43) [GCC 14.3.0]
  Linux-6.8.0-1044-oracle-x86_64-with-glibc2.35

I will try to change it to torch cpu tokenizer, do you have any advice about how to do that?

[acoracle]

I managed to use torch for audio tokenizer with the following configuration

── Audio tokenizer

audio_backend: torch
audio_tokenizer_repo: OpenMOSS-Team/MOSS-Audio-Tokenizer
audio_model_name_or_path: OpenMOSS-Team/MOSS-Audio-Tokenizer
audio_tokenizer_device: cpu

The audio gets generated, but it is still low quality.

[cms42]

Thank you for your detailed feedback ! We are looking into this issue. Would you mind providing your prompt and reference audio (if any) ?

[acoracle]

Sure, this is the command I use to generate
python -m moss_tts_delay.llama_cpp --config configs/llama_cpp/cpu-only.yaml --text "In most organizations, the difference between being simple" --output output.vaw --heads-backend numpy

I don't use reference audio for these tests.

I amm adding also the output of the generation

MOSS-TTS-llama.cpp.output.txt

[cms42]

@acoracle Sorry but we still could not figure out the issue (unable to reproduce on both cpu or gpu) - maybe try the new implementation and see whether it fixes the bug ?

https://github.com/OpenMOSS/llama.cpp/blob/main/docs/moss-tts-firstclass-e2e.md