AI RESEARCH
Which Similarity-Sensitive Entropy (Sentropy)?
arXiv CS.LG
•
ArXi:2511.03849v4 Announce Type: replace-cross Shannon entropy is not the only entropy that is relevant to machine-learning datasets, nor possibly even the most important one. Traditional entropies such as Shannon entropy capture information represented by elements' frequencies but not the richer information encoded by their similarities and differences. Capturing the latter requires similarity-sensitive entropy (``sentropy''). Sentropy can be measured using either the recently developed Leinster-Cobbold-Reeve framework (LCR) or the newer Vendi score (VS.