AI RESEARCH
Elastic-dLLM: Position Preserving Context Compression and Augmentation of Diffusion LLMs
arXiv CS.LG
•
ArXi:2605.18165v1 Announce Type: new Unlike autoregressive models, which generate one token at a time, dLLMs denoise a chunk of [MASK] tokens jointly and sample one or tokens per step; despite enabling parallel decoding, this process incurs substantial computational cost due to the large chunk size of masked tokens. We observe that much of this cost is spent on repeatedly processing the preceding context and many [MASK] tokens with the same feature representations, indicating considerable computational redundancy.