AI RESEARCH
Where Should Diffusion Enter a Language Model? Geometry-Guided Hidden-State Replacement
arXiv CS.CL
•
ArXi:2605.14368v1 Announce Type: new Continuous diffusion language models lag behind autoregressive transformers, partly because diffusion is applied in spaces poorly suited to language denoising and token recovery. We propose DiHAL, a geometry-guided diffusion-transformer hybrid that asks where diffusion should enter a pretrained transformer. DiHAL scores layers with geometry-based proxies, selects a diffusion-friendly hidden-state interface, and replaces the lower transformer prefix with a diffusion bridge while retaining the upper layers and original LM head.