Test-Time Adaptation via Many-Shot Prompting: Benefits, Limits, and Pitfalls

ArXi:2603.05829v1 Announce Type: new Test-time adaptation enables large language models (LLMs) to modify their behavior at inference without updating model parameters. A common approach is many-shot prompting, where large numbers of in-context learning (ICL) examples are injected as an input-space test-time update. Although performance can improve as nstrations are added, the reliability and limits of this update mechanism remain poorly understood, particularly for open-source models.