AI RESEARCH
Sparse Attention Remapping with Clustering for Efficient LLM Decoding on PIM
arXiv CS.LG
•
ArXi:2505.05772v2 Announce Type: replace-cross Transformer-based models are the foundation of modern machine learning, but their execution, particularly during autoregressive decoding in large language models (LLMs), places significant pressure on memory systems due to frequent memory accesses and growing key-value (KV) caches. This creates a bottleneck in memory bandwidth, especially as context lengths increase. Processing-in-memory (PIM) architectures are a promising solution, offering high internal bandwidth and compute parallelism near memory.