Extending Minimal Pairs with Ordinal Surprisal Curves and Entropy Across Applied Domains

ArXi:2603.14400v1 Announce Type: cross The minimal pairs paradigm of comparing model probabilities for contrasting completions has proven useful for evaluating linguistic knowledge in language models, yet its application has largely been confined to binary grammaticality judgments over syntactic phenomena. Additionally, standard prompting-based evaluation requires expensive text generation, may elicit post-hoc rationalizations rather than model judgments, and discards information about model uncertainty.