AI RESEARCH
Beyond Single Tokens: Distilling Discrete Diffusion Models via Discrete MMD
arXiv CS.LG
•
ArXi:2603.20155v1 Announce Type: new It is currently difficult to distill discrete diffusion models. In contrast, continuous diffusion literature has many distillation approaches methods that can reduce sampling steps to a handful. Our method, Discrete Moment Matching Distillation (D-MMD), leverages ideas that have been highly successful in the continuous domain. Whereas previous discrete distillation methods collapse, D-MMD maintains high quality and diversity (given sufficient sampling steps). This is nstrated on both text and image datasets.