AI RESEARCH
Pushing the Boundaries of Multiple Choice Evaluation to One Hundred Options
arXiv CS.CL
•
ArXi:2604.14634v1 Announce Type: new Multiple choice evaluation is widely used for benchmarking large language models, yet near ceiling accuracy in low option settings can be sustained by shortcut strategies that obscure true competence. Therefore, we propose a massive option evaluation protocol that scales the candidate set to one hundred options and sharply reduces the impact of chance performance. We apply this framework to a Korean orthography error detection task where models must pick the single incorrect sentence from a large candidate set.