kv-cache : support attention rotation for heterogeneous iSWA by ggerganov · Pull Request #21513 · ggml-org/llama.cpp
r/LocalLLaMA
•
Generative AI
Open Source AI
Tl;dr: Fixes KV-cache rotation for hybrid-attention models like Gemma 4 (Not actually TurboQuant, but you can call it TurboQuant if that makes you feel better) submitted by /u/jacek2023 [link] [comments]