Deep sequence models tend to memorize geometrically; it is unclear why

ArXi:2510.26745v3 Announce Type: replace Deep sequence models are said to atomic facts predominantly in the form of associative memory: a brute-force lookup of co-occurring entities. We identify a dramatically different form of storage of atomic facts that we term as geometric memory. Here, the model has synthesized embeddings encoding novel global relationships between all entities, including ones that do not co-occur in