AI RESEARCH
Efficient Remote KV Cache Reuse with GPU-native Video Codec
arXiv CS.LG
•
ArXi:2602.09725v3 Announce Type: replace-cross Remote KV cache reuse fetches KV cache for identical contexts from remote storage, avoiding recomputation, accelerating LLM inference. While it excels in high-speed networks, its performance degrades significantly in bandwidth-limited scenarios. Recent studies address this by transmitting KV caches in compressed form, but the associated heavyweight decompression counteracts the KV reuse benefits. In this paper, we propose an efficient and widely deployable remote KV cache reuse solution that leverages GPU-native video codecs.