AI RESEARCH
Online Distributionally Robust LLM Alignment via Regression to Relative Reward
arXiv CS.LG
•
ArXi:2509.19104v2 Announce Type: replace Reinforcement Learning with Human Feedback (RLHF) has become crucial for aligning Large Language Models (LLMs) with human intent. However, existing offline RLHF approaches suffer from overoptimization, where language models degrade by overfitting inaccuracies and drifting from preferred behaviors observed during