AI RESEARCH
Exact Gaussian Moment Matching for Residual Networks: a Second-Order Method
arXiv CS.LG
•
ArXi:2601.22307v2 Announce Type: replace We study the problem of propagating the mean and covariance of a general multivariate Gaussian distribution through a deep (residual) neural network using layer-by-layer moment matching. We close a longstanding gap by deriving exact moment matching for the probit, GeLU, ReLU (as a limit of GeLU), Heaviside (as a limit of probit), and sine activation functions; for both feedforward and generalized residual layers.