AI RESEARCH
Principles and Guidelines for Randomized Controlled Trials in AI Evaluation
arXiv CS.LG
•
ArXi:2605.02050v1 Announce Type: cross This work establishes a foundational framework for standardizing AI evaluation RCTs (sometimes called human uplift studies). Drawing on established experimental practices from disciplines with established RCT traditions, including software engineering, economics, clinical and health sciences, and psychology, we adopt the (Shadish, 2002) four-validity framework and extend it with a fifth principle on transparency, repeatability, and verification adapted from the Transparency and Openness Promotion (TOP) Guidelines (Center for Open Science, 2025.