Easier to Judge than to Find: Predicting In-Context Learning Success for Demonstration Selection

ArXi:2605.18512v1 Announce Type: new In-context learning (ICL) is highly sensitive to which nstrations appear in the prompt, but selecting them is expensive because the space of possible nstration contexts and combinations is enormous. We argue that nstration selection is \emph{easier to judge than to find}: predicting whether a specific query--context pair $(q,D)$ will succeed is cheaper and general than searching for an optimal $D^\star$. Based on this insight, we propose DiSP, a sample-and-judge framework that stratifies queries by difficulty.