in-context learning
in-context-learning-258b8696·3 events·first seen 29d agoAliases: in-context learning
Co-occurring entities
More like this (12)
Recent events (3)
DiSP: A Sample-and-Judge Framework for Efficient In-Context Learning Demonstration Selection
DiSP reframes ICL demonstration selection as a prediction problem rather than a search problem, arguing it is cheaper to judge whether a query-context pair will succeed than to find an optimal context. The framework stratifies queries by difficulty using a lightweight router, trains level-specific judges, and applies stop-on-acceptance judging under an explicit budget. Evaluated on five classification datasets with Llama 3-8B and Qwen 2.5-7B, DiSP improves over strong learned selection baselines by up to 3.4% accuracy while achieving up to 23x wall-clock speedup.
Language models are few-shot learners
OpenAI published the GPT-3 paper introducing a 175-billion-parameter autoregressive language model demonstrating strong few-shot learning capabilities across a wide range of NLP tasks. The work showed that scaling language models dramatically improves task-agnostic, few-shot performance, often matching or exceeding fine-tuned models without any gradient updates. This paper became a foundational milestone in the development of large language models and the modern AI landscape.
In-Context Reward Adaptation for Robust Preference Modeling
This paper proposes In-Context Reward Adaptation (ICRA), a transformer-based framework that infers reward structures from small sets of preference demonstrations at inference time, without retraining. The key finding is that standard transformers exhibit asymptotic bias toward ground-truth rewards, but incorporating human response time as an auxiliary signal resolves this limitation and enables generalization to unseen preference domains. The approach addresses a core limitation of static RLHF reward models, which fail to handle heterogeneous or shifting human value distributions.