AI RESEARCH
The Verification Tax: Fundamental Limits of AI Auditing in the Rare-Error Regime
arXiv CS.LG
•
ArXi:2604.12951v1 Announce Type: new The most cited calibration result in deep learning -- post-temperature-scaling ECE of 0.012 on CIFAR-100 (Guo, 2017) -- is below the statistical noise floor. We prove this is not a failure of the experiment but a law: the minimax rate for estimating calibration error with model error rate epsilon is Theta((Lepsilon/m)^{1/3}), and no estimator can beat it. This "verification tax" implies that as AI models improve, verifying their calibration becomes fundamentally harder -- with the same exponent in opposite directions.