H-Probes: Extracting Hierarchical Structures From Latent Representations of Language Models

ArXi:2605.00847v1 Announce Type: cross Representing and navigating hierarchy is a fundamental primitive of reasoning. Large language models have nstrated proficiency in a wide variety of tasks requiring hierarchical reasoning, but there exists limited analysis on how the models geometrically represent the necessary latent constructions for such thinking. To this end, we develop \textit{H-probes}, a collection of linear probes that extract hierarchical structure, specifically depth and pairwise distance, from latent representations.