AI RESEARCH
An experimental study of KV cache reuse strategies in chunk-level caching systems
arXiv CS.LG
•
ArXi:2603.20218v1 Announce Type: cross Retrieval-augmented generation improves large language models' accuracy by adding relevant retrieved text to the prompt. Chunk level caching (CLC) accelerates inference by precomputing KV caches for these retrieved chunks and reusing them. However, these caches miss cross-attention dependencies between chunks, which can reduce output quality. Several methods try to improve CLC accuracy using different techniques. We make two main contributions.