Building on a LLM Quants Testing Site/Ressource - Sharing a few insights from first month, so you can share your thoughts and wishes for the future.

Wanted to share some insights into a project I am building. The focus is to make it easier to understand how quantization affects open weights model on practical work tasks. For every new model being released it seems like there instantly comes our +200 quantizations released within the first couple of days. This is actually great, but I feel like we somewhat have a transparency gap into what is "good enough" when choosing an LLM quantization.