DiSP

DiSP: A Sample-and-Judge Framework for Efficient In-Context Learning Demonstration Selection

DiSP reframes ICL demonstration selection as a prediction problem rather than a search problem, arguing it is cheaper to judge whether a query-context pair will succeed than to find an optimal context. The framework stratifies queries by difficulty using a lightweight router, trains level-specific judges, and applies stop-on-acceptance judging under an explicit budget. Evaluated on five classification datasets with Llama 3-8B and Qwen 2.5-7B, DiSP improves over strong learned selection baselines by up to 3.4% accuracy while achieving up to 23x wall-clock speedup.