AI RESEARCH
A Derandomization Framework for Structure Discovery: Applications in Neural Networks and Beyond
arXiv CS.LG
•
ArXi:2510.19382v2 Announce Type: replace-cross Understanding the dynamics of feature learning in neural networks (NNs) remains a significant challenge. The work of (Mousavi-Hosseini, 2023) analyzes a multiple index teacher-student setting and shows that a two-layer student attains a low-rank structure in its first-layer weights when trained with stochastic gradient descent (SGD) and a strong regularizer. This structural property is known to reduce sample complexity of generalization.