AI RESEARCH
The Topological Trouble With Transformers
arXiv CS.LG
•
ArXi:2604.17121v1 Announce Type: new Transformers encode structure in sequences via an expanding contextual history. However, their purely feedforward architecture fundamentally limits dynamic state tracking. State tracking -- the iterative updating of latent variables reflecting an evolving environment -- involves inherently sequential dependencies that feedforward networks struggle to maintain.