technique

In-Context Reward Adaptation

techniqueactiveprovisionalin-context-reward-adaptation-44407f1e·1 events·first seen 19d ago

Aliases: In-Context Reward Adaptation

Co-occurring entities

Transformers Reinforcement Learning from Human Feedback human response time in-context learning

More like this (12)

Gradient-Guided Reward Optimization Observe-and-Act Adaptive Context Selection Reward Learning from Comparisons In-Context Multiple Instance Learning in-context learning Using Reward Uncertainty to Induce Diverse Behaviour in Reinforcement Learning Hybrid Reward Advantage Splitting LoRA (Low-Rank Adaptation)Multi-Agent Reinforcement Learning from Delayed Marketplace Feedback for Objective-Weight Adaptation in Three-Sided Dispatch Reinforcement Learning Elicits Contextual Learning of Unseen Language Translation Scaling Laws for Reward Model Overoptimization reward model

Recent events (1)

6arXiv · cs.LG·19d ago·source ↗

In-Context Reward Adaptation for Robust Preference Modeling

This paper proposes In-Context Reward Adaptation (ICRA), a transformer-based framework that infers reward structures from small sets of preference demonstrations at inference time, without retraining. The key finding is that standard transformers exhibit asymptotic bias toward ground-truth rewards, but incorporating human response time as an auxiliary signal resolves this limitation and enables generalization to unseen preference domains. The approach addresses a core limitation of static RLHF reward models, which fail to handle heterogeneous or shifting human value distributions.

Evaluation and Benchmarking Alignment and RLHF Transformers In-Context Reward Adaptation Reinforcement Learning from Human Feedback +2 more