What's the plan? Metrics for implicit planning in LLMs and their application to rhyme generation and question answering

ArXi:2601.20164v2 Announce Type: replace-cross Prior work suggests that language models, while trained on next token prediction, show implicit planning behavior: they may select the next token in preparation to a predicted future token, such as a likely rhyming word, as ed by a prior qualitative study of Claude 3.5 Haiku using a cross-layer transcoder. We propose much simpler techniques for assessing implicit planning in language models. With case studies on rhyme poetry generation and question answering, we nstrate that our methodology easily scales to many models.