AI RESEARCH
Back to Blackwell: Closing the Loop on Intransitivity in Multi-Objective Preference Fine-Tuning
arXiv CS.LG
•
ArXi:2602.19041v2 Announce Type: replace A recurring challenge in preference fine-tuning (PFT) is handling $\textit{intransitive}$ (i.e., cyclic) preferences. Intransitive preferences often stem from either $\textit{(i)}$ inconsistent rankings along a single objective or $\textit{(ii)}$ scalarizing multiple objectives into a single metric. Regardless of their source, the downstream implication of intransitive preferences is the same: there is no well-defined optimal policy, breaking a core assumption of the standard PFT pipeline.