AI RESEARCH
Comparing Developer and LLM Biases in Code Evaluation
arXiv CS.CL
•
ArXi:2603.24586v1 Announce Type: cross As LLMs are increasingly used as judges in code applications, they should be evaluated in realistic interactive settings that capture partial context and ambiguous intent. We present TRACE (Tool for Rubric Analysis in Code Evaluation), a framework that evaluates LLM judges' ability to predict human preferences and automatically extracts rubric items to reveal systematic biases in how humans and models weigh each item.