AI RESEARCH
Information-Theoretic Measures in AI: A Practical Decision Guide
arXiv CS.LG
•
ArXi:2604.23716v1 Announce Type: cross Information-theoretic (IT) measures are ubiquitous in artificial intelligence: entropy drives decision-tree splits and uncertainty quantification, cross-entropy is the default classification loss, mutual information underpins representation learning and feature selection, and transfer entropy reveals directed influence in dynamical systems. A second, less consolidated family of measures, integrated information (Phi), effective information (EI), and autonomy, has emerged for characterizing agent complexity.