Beyond Compromise: Pareto-Lenient Consensus for Efficient Multi-Preference LLM Alignment

ArXi:2604.05965v1 Announce Type: new Transcending the single-preference paradigm, aligning LLMs with diverse human values is pivotal for robust deployment. Contemporary Multi-Objective Preference Alignment (MPA) approaches predominantly rely on static linear scalarization or rigid gradient projection to navigate these trade-offs. However, by enforcing strict conflict avoidance or simultaneous descent, these paradigms often prematurely converge to local stationary points.