AI RESEARCH
Logit-KL Flow Matching: Non-Autoregressive Text Generation via Sampling-Hybrid Inference
arXiv CS.LG
•
ArXi:2411.16821v5 Announce Type: replace-cross Non-autoregressive (NAR) language models offer notable efficiency in text generation by circumventing the sequential bottleneck of autoregressive decoding. However, accurately modeling dependencies in discrete sequences remains challenging in this paradigm. In this work, we advance the field of NAR generation by applying conditional flow matching (CFM) methods grounded in geometrically principled interpolation, specifically leveraging Kullback-Leibler (KL) divergence geodesics, which correspond to linear interpolation in logit space.