Loop, Think, & Generalize: Implicit Reasoning in Recurrent-Depth Transformers

ArXi:2604.07822v1 Announce Type: new We study implicit reasoning, i.e. the ability to combine knowledge or rules within a single forward pass. While transformer-based large language models substantial factual knowledge and rules, they often fail to compose this knowledge for implicit multi-hop reasoning, suggesting a lack of compositional generalization over their parametric knowledge. To address this limitation, we study recurrent-depth transformers, which enables iterative computation over the same transformer layers.