AI RESEARCH
Arena as Offline Reward: Efficient Fine-Grained Preference Optimization for Diffusion Models
arXiv CS.CV
•
ArXi:2605.06070v1 Announce Type: new Reinforcement learning from human feedback (RLHF) effectively promotes preference alignment of text-to-image (T2I) diffusion models. To improve computational efficiency, direct preference optimization (DPO), which avoids explicit reward modeling, has been widely studied. However, its reliance on binary feedback limits it to coarse-grained modeling on chosen-rejected pairs, resulting in suboptimal optimization.