AI RESEARCH
Forget BIT, It is All about TOKEN: Towards Semantic Information Theory for LLMs
arXiv CS.AI
•
ArXi:2511.01202v3 Announce Type: replace-cross Despite the unprecedented empirical triumphs of LLMs across diverse real-world applications, the prevailing research paradigm remains overwhelmingly heuristic and experimentally driven, inextricably tethered to astronomical computational resources and massive data regimes. A rigorous theoretical elucidation of LLMs -- their foundational "first principles" -- remains profoundly elusive.