Efficient Bayesian Inference from Noisy Pairwise Comparisons

ArXi:2510.09333v2 Announce Type: replace Evaluating generative models is challenging because standard metrics often fail to reflect human preferences. Human evaluations are reliable but costly and noisy, as participants vary in expertise, attention, and diligence. Pairwise comparisons improve consistency, yet aggregating them into overall quality scores requires careful modeling. Bradley-Terry-based methods update item scores from comparisons, but existing approaches either ignore rater variability or lack convergence guarantees, limiting robustness and interpretability. We.