A Qualitative Test-Risk Mechanism for Scaling Behavior in Normalized Residual Networks

ArXi:2605.08297v1 Announce Type: cross The scaling behavior, in which test performance often improves as model size and data increase, is a central empirical phenomenon in modern deep learning, yet its theoretical basis remains incomplete. In this paper, we study depth expansion in normalized residual networks: starting from a trained model in an old hypothesis class, we insert a new residual block at an intermediate layer and ask when such an expansion can yield a provable improvement in test risk.