AI RESEARCH
Scaling Laws from Sequential Feature Recovery: A Solvable Hierarchical Model
arXiv CS.LG
•
ArXi:2605.14567v1 Announce Type: cross We propose a simple mechanism by which scaling laws emerge from feature learning in multi-layer networks. We study a high-dimensional hierarchical target that is a globally high-degree function, but that can be represented by a combination of latent compositional features whose weights decrease as a power law.