Expressive Power of Implicit Models: Rich Equilibria and Test-Time Scaling

ArXi:2510.03638v4 Announce Type: replace-cross Implicit models, an emerging model class, compute outputs by iterating a single parameter block to a fixed point. This architecture realizes an infinite-depth, weight-tied network that trains with constant memory, significantly reducing memory needs for the same level of performance compared to explicit models. While it is empirically known that these compact models can often match or even exceed the accuracy of larger explicit networks by allocating test-time compute, the underlying mechanism remains poorly understood.