AI RESEARCH
Criticality and Saturation in Orthogonal Neural Networks
arXiv CS.LG
•
ArXi:2605.06563v1 Announce Type: new It has been known for a long time that initializing weight matrices to be orthogonal instead of having i.i.d. Gaussian components can improve