AI RESEARCH
MMErroR: A Benchmark for Erroneous Reasoning in Vision-Language Models
arXiv CS.LG
•
ArXi:2601.03331v2 Announce Type: replace-cross Recent advances in Vision-Language Models (VLMs) have improved performance in multi-modal learning, raising the question of whether these models truly understand the content they process. Crucially, can VLMs detect when a reasoning process is wrong and identify its error type? To answer this, we present MMErroR, a multi-modal benchmark of 1997 samples, each embedding a single coherent reasoning error. These samples span 24 subdomains across six top-level domains, ensuring broad coverage and taxonomic richness.