Qwen3.6 27B's surprising KV cache quantization test results (Turbo3/4 vs F16 vs Q8 vs Q4)
r/LocalLLaMA
•
Generative AI
Open Source AI
AI Research
I've been using Qwen3.6-27B-Q5_K_M with turbo3 KV cache since it's been released, and I haven't had any issues at all (no loops, no memory loss, etc.). However, I'm also aware that V cache compression is not really recommended in most cases. So I wanted to check how it is possible and I learned that llama-perplexity.exe is the right tool for this test. I'm using TheTom's turboquant_plus built on my machine - AFAIK you can download a pre-built release by now as well. I have a 3090 eGPU and using 200k context.