Do Transformers Use their Depth Adaptively? Evidence from a Relational Reasoning Task

ArXi:2604.12426v1 Announce Type: new We investigate whether transformers use their depth adaptively across tasks of increasing difficulty. Using a controlled multi-hop relational reasoning task based on family stories, where difficulty is determined by the number of relationship hops that must be composed, we monitor (i) how predictions evolve across layers via early readouts (the logit lens) and (ii) how task-relevant information is integrated across tokens via causal patching.