MiniMax M2.7 GGUF Investigation, Fixes, Benchmarks
r/LocalLLaMA
•
Generative AI
Open Source AI
AI Tools
Hey r/LocalLLaMA, we did an investigation into MiniMax-M2.7 GGUF causing NaNs on perplexity. Our findings show the issue affects 21%-38% of all GGUFs on Hugging Face (not just ours). Other popular community uploaders have 38% (10/26) NaNs, another deleted theirs (1/4), and 22% of ours had NaNs (5/23) - we fixed ours. When running 99.9% KLD and other metrics, all are fine. We found overflowing in llama.cpp to be the culprit. We did PPL, KLD 99.9% benchmarks as well - lower left is better. Perplexity NaNs during block 32 - this was also found by the community and other quant uploaders.