Skip to content

[WIP] feat: ExLlamaV3 quantization format#1398

Draft
AlpinDale wants to merge 1 commit into
mainfrom
exl3
Draft

[WIP] feat: ExLlamaV3 quantization format#1398
AlpinDale wants to merge 1 commit into
mainfrom
exl3

Conversation

@AlpinDale
Copy link
Copy Markdown
Collaborator

Just some tests in running exl3 models. Currently kernels produce NaN so it's nowhere near ready, but weight loading works.

@AlpinDale AlpinDale marked this pull request as draft July 22, 2025 22:23
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant