Entity · paper

Entropy Is Not Enough: Unlocking Effective Reinforcement Learning for Visual Reasoning via Vision-Anchored Token Selection

paperactive

entropy-is-not-enough-unlocking-effective-reinforcement-learning-for-visual-reasoning-via-vision-anchored-token-selection-304a13e4

·1 events·first seen Jun 3, 2026

Aliases: Entropy Is Not Enough: Unlocking Effective Reinforcement Learning for Visual Reasoning via Vision-Anchored Token Selection

Co-occurring entities

VEPO

More like this (12)

Visually Grounded Self-Reflection for Vision-Language Models via Reinforcement Learning Using Reward Uncertainty to Induce Diverse Behaviour in Reinforcement Learning Entropy-Regularized Reinforcement Learning Learning to Reason by Analogy via Retrieval-Augmented Reinforcement Fine-Tuning FORCE: Efficient VLA Reinforcement Fine-Tuning via Value-Calibrated Warm-up and Self-Distillation UniIntervene: Agentic Intervention for Efficient Real-World Reinforcement Learning Self-Refining Agentic Reinforcement Learning for Vision-Conditioned UAV Navigation Reroute, Don't Remove: Recoverable Visual Token Routing for Vision-Language Models Physics-EnhAnced Reinforcement Learning Token Budget Saturation and Mechanistic Early Detection of Reasoning Non-Convergence in Chain-of-Thought Models Learning Process Rewards via Success Visitation Matching for Efficient RL SERPO: Self-Evolving Rubric Policy Optimization for Open-Ended Test-Time Reinforcement Learning

Recent events (1)

5arXiv · cs.AI·Jun 3, 2026·source ↗

VEPO: Vision-anchored token selection improves RL for visual reasoning

A new arXiv paper identifies a failure mode of entropy-based credit assignment in multimodal reinforcement learning: vision-sensitive tokens with naturally low entropy are systematically ignored, causing the mechanism to collapse in visual reasoning tasks. The authors propose VEPO (Vision-Entropy token-selection for Policy Optimization), which couples visual sensitivity with token entropy via a multiplicative scheme to redirect gradient credit toward tokens that are both visually grounded and semantically informative. VEPO outperforms entropy-only baselines by 2.28 points at 7B scale and 3.15 points at 3B scale on visual reasoning benchmarks.

Alignment and RLHF Multimodal Progress VEPO Entropy Is Not Enough: Unlocking Effective Reinforcement Learning for Visual Reasoning via Vision-Anchored Token Selection