Learning What Evaluators Value: A Reliable Approach to Modeling Evaluator Preferences

ArXi:2605.16615v1 Announce Type: new In many applications, human and LLM evaluators use assessments of relevant criteria to create an overall evaluation for an item or individual. For example, in admissions, committees assess candidates on attributes such as test scores, GPA, and research experience to evaluate their overall fit for the program. Another example arises in medical care where clinicians use patient reports of symptoms to consider preliminary diagnoses and assess risks.