AI RESEARCH
Tensor Product Representation Probes Reveal Shared Structure Across Linear Directions
arXiv CS.LG
•
ArXi:2605.09967v1 Announce Type: new While researchers are finding concepts represented as linear directions in language models, a bag of linear directions fails to capture relational structure. To better understand this dichotomy, we study a model with known linear representations, but trained in a highly structured domain -- the board game Othello. While the model's internal board-state representation is linearly decodable, we find additional structure in the form of tensor product representations (TPRs.