AI RESEARCH
Before You Interpret the Profile: Validity Scaling for LLM Metacognitive Self-Report
arXiv CS.CL
•
ArXi:2604.17707v1 Announce Type: new Clinical personality assessment screens response validity before interpreting substantive scales. LLM evaluation does not. We apply the validity scaling framework from the PAI and MMPI-3 to metacognitive probe data from 20 frontier models across 524 items. Six validity indices are operationalised: L (maintaining confidence on errors), K (betting on errors), F (withdrawing consensus-endorsed items), Fp (withdrawing correct answers), RBS (inverted monitoring), and TRIN (fixed responding.