AI RESEARCH
A Gaussian Process View on Observation Noise and Initialization in Wide Neural Networks
arXiv CS.LG
•
ArXi:2502.01556v3 Announce Type: replace Performing gradient descent in a wide neural network is equivalent to computing the posterior mean of a Gaussian Process with the Neural Tangent Kernel (NTK-GP), for a specific prior mean and with zero observation noise. However, existing formulations have two limitations: (i) the NTK-GP assumes noiseless targets, leading to misspecification on noisy data; (ii) the equivalence does not extend to arbitrary prior means, which are essential for well-specified models. To address (i), we