AI RESEARCH
A Game Theoretic Free Energy Analysis of Higher Order Synergy in Attention Heads of Large Language Models
arXiv CS.AI
•
ArXi:2605.09515v1 Announce Type: new Large language models rely on multihead attention, but interactions among heads remain poorly understood. We apply the Game Theoretic Free Energy Principle (GTFEP): a framework casting multiagent systems as distributed variational inference to analyze attention heads as bounded rational agents. According to GTFEP, each head minimizes its variational free energy, and collective behavior follows a Gibbs distribution over coalition structures whose energy is decomposed into Harsanyi dividends.