Discussions

• 1 You must be logged in to vote

  💡

  Pass whole dataset/and subsequent inputs/ to each expert, not only routed data.

  avtc started Nov 25, 2025 in Ideas

  

  

  11
• 1 You must be logged in to vote

  🙌

  offload_to_disk=True is very slow for second initial forward and uses more VRAM

  avtc started Nov 4, 2025 in Show and tell

  

  

  0
• 1 You must be logged in to vote

  🙏

  Configuring group_size > 128

  hzfantasy asked Jun 20, 2025 in Q&A · Closed · Unanswered

  

  

  

  

  3
• 1 You must be logged in to vote

  🙏

  help to start

  kalle07 asked Oct 4, 2025 in Q&A · Unanswered

  

  

  0
• 1 You must be logged in to vote

  🙏

  Is this library much slower than AutoGPTQ?

  CHNtentes asked Jan 2, 2025 in Q&A · Closed · Answered

  

  

  13
• 2 You must be logged in to vote

  🙏

  PyPi v3.0.0 release

  ankandrew asked May 17, 2025 in Q&A · Closed · Unanswered

  

  

  

  

  2
• 1 You must be logged in to vote

  🙏

  Setting activation bit width

  jeyapradhaj asked Jul 3, 2025 in Q&A · Closed · Unanswered

  

  

  1
• 2 You must be logged in to vote

  🙏

  I've made over 20 tests and GPTQv2 is always worse than BitsandBytes 4bit even the 8bit GTPQv2 is worse.

  TByte007 asked May 9, 2025 in Q&A · Unanswered

  

  

  0
• 1 You must be logged in to vote

  💡

  Add conversion from ModelCloud Quantizations (GPTQ, GPTQ-v2, QQQ + Rotation) to GGUF

  joseph777111 started Apr 15, 2025 in Ideas

  

  

  6
• 2 You must be logged in to vote

  🙏

  why load gptq int4 model by default with torch_dtype=auto is always use float16?

  zzh-www asked Mar 5, 2025 in Q&A · Closed · Answered

  

  

  16
• 0 You must be logged in to vote

  📣

  Welcome to GPTQModel Discussions!

  Qubitium announced Dec 16, 2024 in Announcements

  

  

  0