Adaptive Loops and Memory in Transformers: Think Harder or Know More?

ArXi:2603.08391v1 Announce Type: new Chain-of-thought (CoT) prompting enables reasoning in language models but requires explicit verbalization of intermediate steps. Looped transformers offer an alternative by iteratively refining representations within hidden states. This parameter efficiency comes at a cost, as looped models lack the storage capacity of deeper models which use unique weights per layer.