AI RESEARCH
SGD for Variational Inference: Tackling Unbounded Variance via Preconditioning and Dynamic Batching
arXiv CS.LG
•
ArXi:2605.07531v1 Announce Type: new Black-Box Variational Inference (BBVI) typically relies on Stochastic Gradient Descent (SGD) to optimize the Evidence Lower Bound (ELBO). However, the stochastic gradients in BBVI inherently exhibit unbounded variance, violating standard assumptions and instead satisfying the weaker Blum-Gladyshe (BG) condition, where variance grows quadratically with distance from the optimum. In this paper, we bridge the gap between stochastic optimization theory and the practical instances of