AI RESEARCH
"Noisier" Noise Contrastive Eestimation is (Almost) Maximum Likelihood
arXiv CS.LG
•
ArXi:2405.16730v2 Announce Type: replace Noise Contrastive Estimation (NCE) has fueled major breakthroughs in representation learning and generative modeling. Yet a long-standing challenge remains: accurately estimating ratios between distributions that differ substantially, which significantly limits the applicability of NCE on modern high-dimensional and multimodal datasets. We revisit this problem from a less explored perspective: the magnitude of the noise distribution.