Tail-Aware Information-Theoretic Generalization for RLHF and SGLD

ArXi:2604.10727v1 Announce Type: cross Classical information-theoretic generalization bounds typically control the generalization gap through KL-based mutual information and. therefore. rely on boundedness or sub-Gaussian tails via the moment generating function (MGF). In many modern pipelines, such as robust learning, RLHF, and stochastic optimization, losses and rewards can be heavy-tailed, and MGFs may not exist, rendering KL-based tools ineffective.