Self-Consolidating Language Models: Continual Knowledge Incorporation from Context

ArXi:2605.07076v1 Announce Type: cross Large language models (LLMs) increasingly receive information as streams of passages, conversations, and long-context workflows. While longer context windows expose evidence, they do not ensure that useful information is preserved and reused. We study continual context consolidation: writing current context into model weights while limiting interference with previously consolidated information. We propose \textbf{S}elf-\textbf{Co}nsolidating \textbf{L}anguage Models (SCoL), a post.