In-Context Fixation: When Demonstrated Labels Override Semantics in Few-Shot Classification

ArXi:2605.08295v1 Announce Type: cross While random nstration labels barely hurt in-context learning (Min, 2022), we show that homogeneous labels--even semantically valid ones--collapse accuracy to <=12% across six models (Pythia, Llama, Qwen; 0.8B--8B) and four tasks. The trigger is label-slot content: the model treats tokens occupying the label position as an exhaustive answer vocabulary, with homogeneity as the maximally collapsed case.