[Feature Request] Use llama.cpp directly on the lxc without docker

[eikaramba]

Describe the feature

i adapted the great script from https://github.com/kyuz0/amd-strix-halo-toolboxes/blob/main/toolboxes/Dockerfile.rocm-7.2 (I used the 7.1.1 script which is now removed) to compile and use llama.cpp direclty on the lxc without docker. if anyone is interested here it is:
https://github.com/eikaramba/proxmox-setup-scripts/blob/main/lxc/install-rocm-llamacpp-in-lxc.sh

Maybe this is something that is of interest and can be adopted :) feel free to close this "issue" if the repo should stick to the bare minimum and not include application specific scripts.

will see if i can update it soon. but i wanted to say it works, at least for super simple "hello world" messages. still waiting for newer proxmox kernels above 6.17 before doing more testing with rocm 7.2 see compatibility chart here

Additional information

• Would you be willing to help implement this feature?

Final checks

• I understand that this is a feature request function. I also understand that if i use this to report a bug report, that my issue might be un-answered closed.

[jammsen]

Yeah i read the comments of @phueper over here ollama/ollama#13000 (comment) - I would rather wait for something a little more stable to include. Im also not sure if only 6.18 and 7.2 have a problem and 7.2+6.17 works good.

But yes, the option in the create gpu-lxc script for pick and choose docker or llamp.cpp i think is a valid reason to add this to the script too. Some people like to keep the virtualization layers to a minimum and try to run as natively as possible.

[phueper]

@jammsen not sure if it helps but i managed to get ROCm 7.2 working with the Ubuntu 25.10 default 6.17 kernel, but needed to use a firmware package from 26.04 and the amdgpu dkms driver... This combination seems pretty stable currently, but certainly isn't out of the box 😊.. some Infos on what i did are here: ROCm/ROCm#5905 (comment)

[jammsen]

@jammsen not sure if it helps but i managed to get ROCm 7.2 working with the Ubuntu 25.10 default 6.17 kernel, but needed to use a firmware package from 26.04 and the amdgpu dkms driver... This combination seems pretty stable currently, but certainly isn't out of the box 😊.. some Infos on what i did are here: ROCm/ROCm#5905 (comment)

Thanks Pattrick, you rock! ❤️

[jammsen]

Hey @eikaramba
As we know now so much more about the ROCM 7.2 and Linux Kernel problems, i would wait for ROCM 7.2 and PVE+6.18.X Kernel works really good.

For now i consider 7.1.1 with PVE 9.1.5 and 6.17.X as stable. Dont want to jinx/break other users stuff.

Hope you can understand this, delay.