A quantitative analysis of semantic information in deep representations of text and images

ArXi:2505.17101v5 Announce Type: replace-cross It was recently observed that the representations of different models that process identical or semantically related inputs tend to align. We analyze this phenomenon using the Information Imbalance, an asymmetric rank-based measure that quantifies the capability of a representation to predict another, providing a proxy of the cross-entropy which can be computed efficiently in high-dimensional spaces.