Tool for Creating Your Own High-Quality GGUF Quants (Docs + Web UI)
r/LocalLLaMA
•
Generative AI
Open Source AI
AI Research
For anyone interested in building their own GGUF quants, I’ve put together the GGUF-Tool-Suite docs and a simple web UI to make the process easier. Docs: Web UI: The goal is to let anyone benchmark and automatically produce GGUFs of any size for ik_llama.cpp and llama.cpp, either through the web UI or the CLI. The tool suite has already been adopted by a few passionate users looking for better GGUF quality and flexibility to fit hardware optimally.