Escaping the Agreement Trap: Defensibility Signals for Evaluating Rule-Governed AI

ArXi:2604.20972v1 Announce Type: new Content moderation systems are typically evaluated by measuring agreement with human labels. In rule-governed environments this assumption fails: multiple decisions may be logically consistent with the governing policy, and agreement metrics penalize valid decisions while mischaracterizing ambiguity as error -- a failure mode we term the Agreement Trap. We formalize evaluation as policy-grounded correctness and