AI RESEARCH
AsyncTLS: Efficient Generative LLM Inference with Asynchronous Two-level Sparse Attention
arXiv CS.CL
•
ArXi:2604.07815v1 Announce Type: new Long-context inference in LLMs faces the dual challenges of quadratic attention complexity and prohibitive KV cache memory. While token-level sparse attention offers superior accuracy, its indexing overhead is costly; block-level methods improve efficiency but sacrifice precision.