AI RESEARCH
Hierarchical vs. Flat Iteration in Shared-Weight Transformers
arXiv CS.AI
•
ArXi:2604.14442v1 Announce Type: cross We present an empirical study of whether hierarchically structured, shared-weight recurrence can match the representational quality of independent-layer stacking in a Transformer-based language model. HRM-LM replaces L independent Transformer layers with a two-speed recurrent pair: a Fast module operating at every step for local refinement, and a Slow module operating every T steps for global compression. This recurrent hierarchy is unrolled for M = N x T steps with shared parameters.