ALIEN: Aligned Entropy Head for Improving Uncertainty Estimation of LLMs

ArXi:2505.15443v2 Announce Type: replace Uncertainty estimation remains a key challenge when adapting pre-trained language models to downstream classification tasks, with overconfidence often observed for difficult inputs. While predictive entropy provides a strong baseline for uncertainty estimation, it considers mainly aleatoric uncertainty and has limited capacity to capture effects, such as class overlap or ambiguous linguistic cues. We