AI RESEARCH
Efficient Long-Context Modeling in Diffusion Language Models via Block Approximate Sparse Attention
arXiv CS.CV
•
ArXi:2605.19726v1 Announce Type: new Diffusion Language Models (DLMs) enable globally coherent, bidirectional, and controllable text generation, offering advantages over traditional autoregressive LLMs, while scaling to ultra-long sequences remains costly. Many existing block-sparse attention methods select blocks by fixed sampling patterns over the high-resolution attention space, such as tail regions or anti-diagonal stripes. Such prior-driven sampling can miss salient tokens and.