Got a model in mind? Drop a comment and we’ll work on losslessly compressing it!

[tonyzhang617]

We’ve been rolling out losslessly compressed versions of popular models using DFloat11, cutting down size by ~30% while keeping outputs bit-for-bit identical.

Have a model you want to see compressed next? Drop it in the comments.

[Taikakim]

I'd love to see Stable Audio Open!

[tonyzhang617]

Thanks for the suggestion! Just a heads-up, Stable Audio Open looks like it's in FP16, while DF11 is designed for BF16 models. We can convert FP16 to BF16 and then compress it with DF11, but that would introduce some minor precision loss. Would you still like to try out the model?

[Taikakim]

Hi, yes I think they trained in FP16, personally I fine tune the model if FP16-mixed or BF16-mixed, not sure if that is wise though since they have a different precision from the original 🤔 But yes, I would be still very interested to try out the compressed version. The current implementation at least in the Gradio GUI of Stable Audio Open does not support batching, and I understood that without it there is a big speed loss? But I think I might be able to hack support in with Claude.

[henilshah06]

Can we have a DF11 version for distilbert-base-uncased?
I converted it from FP32 to BF16 but not sure how to compress it with DF11.
If we can have a BF16 to DF11 conversion code added to the repo, that'd be great.