Correcting Performance Estimation Bias in Imbalanced Classification with Minority Subconcepts

ArXi:2604.26024v1 Announce Type: cross Class-level evaluation can conceal substantial performance disparities across subconcepts within the same class, causing models that perform well on average to fail on specific subpopulations. Prior work has shown that common evaluation measures for imbalanced classification are biased toward larger minority subconcepts and that utility-based reweighting using true subconcept labels can mitigate this bias; however, such labels are rarely available at test time. We.