Why do instructions degrade in long-context LLM conversations, but constraints seem to hold?

r/LocalLLaMA
Generative AI

Observation from working with local LLMs in longer conversations. When designing prompts, most approaches focus on adding instructions: - follow this structure - behave like X - include Y, avoid Z This works initially, but tends to degrade as the context grows: - constraints weaken - verbosity increases - responses drift beyond the task This happens even when the original instructions are still inside the context window. What seems stable in practice is not adding instructions, but.