AI RESEARCH
Beyond Scores: Diagnostic LLM Evaluation via Fine-Grained Abilities
arXiv CS.AI
•
ArXi:2604.12191v1 Announce Type: new Current evaluations of large language models aggregate performance across diverse tasks into single scores. This obscures fine-grained ability variation, limiting targeted model improvement and ability-guided selection for specific tasks. Motivated by this gap, we propose a cognitive diagnostic framework that estimates model abilities across multiple fine-grained dimensions. For mathematics, we construct a 35-dimensional ability taxonomy grounded in cognitive theory and domain knowledge.