AI RESEARCH
Head Forcing: Long Autoregressive Video Generation via Head Heterogeneity
arXiv CS.CV
•
ArXi:2605.14487v1 Announce Type: new Autoregressive video diffusion models real-time synthesis but suffer from error accumulation and context loss over long horizons. We discover that attention heads in AR video diffusion transformers serve functionally distinct roles as local heads for detail refinement, anchor heads for structural stabilization, and memory heads for long-range context aggregation, yet existing methods treat them uniformly, leading to suboptimal KV cache allocation. We propose Head Forcing, a.