AI RESEARCH
UNA: A Unified Supervised Framework for Efficient LLM Alignment Across Feedback Types
arXiv CS.LG
•
ArXi:2408.15339v4 Announce Type: replace RL alignment methods, including RLHF and DPO, are primarily based on pairwise preference data. Although scalar or score-based feedback has been collected in some settings, it is rarely used directly, and preference magnitude information is typically ignored. Furthermore, current alignment frameworks offer limited capability for unifying heterogeneous supervision signals, making it difficult to jointly leverage diverse data types within a single