AI RESEARCH
Connecting Jensen-Shannon and Kullback-Leibler Divergences: A New Bound for Representation Learning
arXiv CS.LG
•
ArXi:2510.20644v2 Announce Type: replace Mutual Information (MI) is a fundamental measure of statistical dependence widely used in representation learning. While direct optimization of MI via its definition as a Kullback-Leibler divergence (KLD) is often intractable, many recent methods have instead maximized alternative dependence measures, most notably, the Jensen-Shannon divergence (JSD) between joint and product of marginal distributions via discriminative losses. However, the connection between these surrogate objectives and MI remains poorly understood.