Entity · paper

A Learning-Rate-Gated Failure of GRPO in a Small Language and Vision-Language Model Web Agent

paperactiveprovisionala-learning-rate-gated-failure-of-grpo-in-a-small-language-and-vision-language-model-web-agent-8120ee4b·1 events·first seen 6d ago

Aliases: A Learning-Rate-Gated Failure of GRPO in a Small Language and Vision-Language Model Web Agent

Co-occurring entities

GRPO (Group Relative Policy Optimization)Set-of-Marks

More like this (12)

Visually Grounded Self-Reflection for Vision-Language Models via Reinforcement Learning TempoVLA: Learning Speed-Controllable Vision-Language-Action Policies Self-Compacting Language Model Agents On the Limits of Prompt-Conditioned Language Models as General-Purpose Learners large language model agents LabVLA: Grounding Vision-Language-Action Models in Scientific Laboratories Uncertainty Quantification for Computer-Use Agents: A Benchmark across Vision-Language Models and GUI Grounding Datasets Towards Continual Motion-Language Agents: LoRA Variants for Incremental Motion Understanding and Generation Does VLA Even Know the Basics? Measuring Commonsense and World Knowledge Retention in Vision-Language-Action Models Neural Collapse Is Forbidden: Information Floors in Language Models Evolution of Accuracy and Visual-Cognitive Errors in a Decade of Vision-Language AI Models visual language model

Recent events (1)

6arXiv · cs.CL·6d ago·source ↗

GRPO fails to improve small web agents when supervised baseline is near-ceiling, study finds

A controlled ablation study across 18 runs tests whether GRPO reinforcement learning adds capability to 4B–8B scale language and vision-language model web agents on top of a strong supervised baseline. The result is a credible null: GRPO does not improve success rates when the supervised model has largely mastered the task distribution, and moderate-to-high learning rates actively degrade text-track performance. The authors identify the mechanism — GRPO only helps when sampling headroom exists (sampled policy succeeds more than greedy), and failure modes dissociate into attention/MLP degradation versus full collapse regimes. Effective rank in late layers tracks capability at 4B but not 8B, flagging a scale-dependent coupling.

Evaluation and Benchmarking Agent and Tool Ecosystem A Learning-Rate-Gated Failure of GRPO in a Small Language and Vision-Language Model Web Agent GRPO (Group Relative Policy Optimization)Set-of-Marks +1 more