6arXiv cs.CL (Computation and Language)·
OSReward: Benchmark and reward models for evaluating VLM judges on computer-use agent trajectories
Researchers introduce OSReward, a benchmark for evaluating vision-language model (VLM) judges on computer-using agent (CUA) trajectories, finding that even state-of-the-art models exhibit systematic leniency bias — mislabeling failed runs as successes. The study also releases OSReward-Hard and OSReward-Multi challenge sets, plus OS-Shepherd-100K, an open corpus of reasoning-annotated trajectory judgments. Built on this corpus, OS-Shepherd (9B and 35B) reward models are trained to match commercial judge quality at 30–60% lower cost. The work addresses a critical gap in scalable CUA evaluation infrastructure needed for both RL training and data curation.
Related guides (3)
Related events (8)
RoboReward: Vision-Language Reward Models for Robot Training via RL
Researchers at Stanford and UC Berkeley developed RoboReward, a family of 4B and 8B vision-language reward models designed to provide reward signals for robot reinforcement learning across diverse robot types and tasks. The team built a novel dataset by augmenting successful robot demonstrations with synthetically generated failure examples using GPT-5 mini and Qwen3-4B, then fine-tuned Qwen3-VL models to predict task progress scores. RoboReward 8B outperformed GPT-5, GPT-5 mini, and Gemini Robotics-ER 1.5 on the new RoboRewardBench evaluation, and in real-world robot trials substantially exceeded prior reward model baselines while still falling short of human-assigned rewards. The authors also release RoboRewardBench as a community benchmark for reward model evaluation.
Mitigating Perceptual Judgment Bias in Multimodal LLM-as-a-Judge via Perceptual Perturbation and Reward Modeling
This paper identifies and analyzes 'Perceptual Judgment Bias' in multimodal LLM judges, where models anchor on response text rather than visual evidence when the two conflict. The authors introduce a Perceptually Perturbed Judgment Dataset using counterfactual responses to isolate perceptual errors, and a training framework combining GRPO-based reward modeling with batch-ranking objectives. Experiments on MLLM-as-a-Judge benchmarks show improved perceptual fidelity, ranking coherence, and alignment with human evaluation.
CORA: Consistency-Oriented Reasoning Alignment addresses thinking-answer gap in multimodal RLVR
Researchers identify and analyze a systematic inconsistency between reasoning traces and final answers in RLVR-trained large vision-language models, showing the problem persists throughout GRPO training and inference. They propose CORA, which introduces a lightweight plug-and-play consistency reward model and a Hybrid Reward Advantage Splitting (HRAS) mechanism to coordinate task and consistency optimization. Experiments across multimodal reasoning benchmarks show CORA improves both task performance and reasoning faithfulness.
Benchmarking LLM judges for citation verification in deep-research systems
A new arXiv paper evaluates 8 LLM judges from 3 model families on citation quality assessment for deep-research systems, testing across 1,248 rubric decisions with human-reviewed gold labels. The study finds that cheaper models remain competitive with frontier models — GPT-5-mini achieves the strongest source-relevance F1 at 0.908 — but judges differ substantially in directional bias (pass-rate drift, false positive/negative rates) even when scalar F1 scores are similar. The key finding is that scalar F1 obscures biases that would be directly reinforced in an RL training loop, making judge calibration a prerequisite before using citation rubrics as reward signals.
QVal: Training-free benchmark for evaluating dense supervision signals in long-horizon LLM agents
QVal is a new training-free testbed for evaluating dense supervision signals used to guide LLM agents over long-horizon trajectories, where outcome-only rewards are too sparse. The framework measures 'Q-alignment' — whether a method's step scores match Q-values from a strong reference policy — enabling comparison of 21 methods across 4 environments and 7 methodological families without running full training pipelines. A key finding is that simple prompting baselines consistently outperform more sophisticated dense supervision methods from recent literature. The benchmark covers over 1,200 evaluation experiments across six open-weight model backbones.
BabelJudge: Benchmark for measuring LLM-as-a-judge reliability across languages and agent trajectories
BabelJudge is a new open-source benchmark and audit framework that systematically measures four failure modes in LLM-as-a-judge systems: position bias, verbosity bias, order inconsistency, and cross-lingual degradation. The framework uses a 'gold-labelling by degradation' technique to generate labeled evaluation pairs without human annotation. Evaluation of Qwen2.5-7B-Instruct-4bit across English, Hindi, Arabic, and Swahili reveals severe cross-lingual reliability drops, with Swahili order consistency collapsing to near-random (0.480). The framework is extended to agentic evaluation with nine trajectory-level perturbations and three new metrics, released as a Python package supporting 11 judge backends.
OCR-Robust benchmark evaluates VLM robustness to visual perturbations on OCR-reasoning tasks
Researchers introduce OCR-Robust, a benchmark of 812 samples designed to evaluate how vision-language models handle OCR-reasoning tasks under controlled visual degradation. The benchmark covers documents, scene text, charts, geometry, and tables, applying 5 perturbation types at 3 severity levels each, and evaluates 18 models using metrics including Relative Corruption Retention and a composite Corruption Robustness Index. Key findings show that higher clean accuracy does not guarantee robustness, and that chart and table inputs are substantially more fragile under perturbation than document-like inputs.
Audit reveals protocol-level shortcuts in large audio-language model judges for speech evaluation
Researchers audit 'protocol-level shortcuts' in large audio-language models (LALMs) used as automatic judges for speech evaluation, testing across three deployment protocols: feature-blueprint judging, reference-conditioned judging, and pairwise A/B comparison. Across six judges and four attributes, several LALMs are found to rely on shortcuts rather than actual audio content — for example, incorrect specialist labels collapse emotion accuracy to 0.10 or below for five judges, and Qwen3-Omni-Thinking shows position bias in A/B comparisons. The findings indicate that high human-agreement scores can overstate judge validity, and that each model-protocol pair requires its own shortcut probe to be trustworthy.