AI RESEARCH
Using LLM-as-a-Judge/Jury to Advance Scalable, Clinically-Validated Safety Evaluations of Model Responses to Users Demonstrating Psychosis
arXiv CS.AI
•
ArXi:2604.02359v1 Announce Type: cross General-purpose Large Language Models (LLMs) are becoming widely adopted by people for mental health. Yet emerging evidence suggests there are significant risks associated with high-frequency use, particularly for individuals suffering from psychosis, as LLMs may reinforce delusions and hallucinations. Existing evaluations of LLMs in mental health contexts are limited by a lack of clinical validation and scalability of assessment.