AI RESEARCH
Where Matters More Than What: Decoding-aligned KV Cache Compression via Position-aware Pseudo Queries
arXiv CS.CL
•
ArXi:2603.11564v1 Announce Type: new The Key-Value (KV) cache is crucial for efficient Large Language Models (LLMs) inference, but excessively long contexts drastically increase KV cache memory footprint. Existing KV cache compression methods typically rely on input-side attention patterns within a prompt observation window to estimate token importance during the prefill stage. They fail to preserve critical tokens for future generation since these assessments are not derived from the decoding process.