AI RESEARCH
Beyond ReinMax: Low-Variance Gradient Estimators for Discrete Latent Variables
arXiv CS.LG
•
ArXi:2603.08257v1 Announce Type: cross Machine learning models involving discrete latent variables require gradient estimators to facilitate backpropagation in a computationally efficient manner. The most recent addition to the Straight-Through family of estimators, ReinMax, can be viewed from a numerical ODE perspective as incorporating an approximation via Heun's method to reduce bias, but at the cost of high variance. In this work, we