AI RESEARCH
Lost in State Space: Probing Frozen Mamba Representations
arXiv CS.LG
•
ArXi:2605.00253v1 Announce Type: cross Mamba's recurrent state h_t is, by construction, a compressed summary of every token seen so far. This raises a tempting hypothesis: if we extract token-level outputs y_t at fixed patch boundaries, we obtain semantic sentence summaries for free, with no pooling head, no fine-tuning, and no [CLS] token. We test this hypothesis carefully.