4arXiv cs.CL (Computation and Language)·
R^3: Curriculum reinforcement learning framework for automated ad compliance rectification in video
Researchers propose R^3, a framework for automatically rectifying textual violations in video advertisements (speech transcripts and on-screen text) while preserving advertiser semantic intent. The system combines a group-relative compliance experience extractor for data synthesis, a curriculum reinforcement learning strategy with hierarchical rewards, and an end-to-end video rectification pipeline covering text recognition, rewriting, and re-rendering. Experiments on industrial datasets and A/B testing show R^3 outperforms baselines on the compliance-vs-intent trade-off.
Related guides (1)
Related events (8)
VRRL: Reinforcement learning framework for visually grounded self-reflection in vision-language models
Researchers propose VRRL, a reinforcement learning training framework designed to improve self-reflection in large vision-language models (LVLMs) by ensuring corrections are grounded in visual inputs rather than purely textual reasoning. The framework uses two mechanisms: random masking of trajectory prefixes to emphasize error recovery, and buffered roll-ins from an experience replay buffer to expose models to diverse failure states. Evaluated on visual grounding tasks (tables, charts) and spatial navigation benchmarks, VRRL substantially improves out-of-distribution accuracy over standard RL and reflection-oriented fine-tuning baselines. The work addresses a concrete failure mode where LVLMs lose visual grounding during chain-of-thought reflection.
RiVER framework enables RL training of LLMs on tasks without ground-truth solutions
Researchers introduce RiVER (Ranking-induced VERifiable framework), a reinforcement learning approach that trains LLMs on score-based optimization tasks using deterministic execution feedback as continuous rewards, without requiring ground-truth answers. The method addresses two failure modes in group-relative RL with continuous rewards—scale dominance and frequency dominance—via calibrated, instance-wise reward shaping. Applied to Qwen3-8B and GLM-Z1-9B-0414 on competitive programming tasks, RiVER improves ALE rating rank by ~9% and also transfers to exact-solution benchmarks (LiveCodeBench, USACO) with 2-4% absolute gains, unlike raw-score baselines. The result suggests score-based heuristic tasks can serve as general-purpose RL training environments for coding ability.
SVR: Self-Verifying Refinement enables adaptive test-time compute via joint verdict-confidence RL
Researchers introduce Self-Verifying Refinement (SVR), a multi-turn reinforcement learning framework that trains language models to self-verify their own outputs and adaptively stop refinement without external verifiers. The model produces a solution, a correctness verdict, and a confidence score at each turn, continuing refinement only when uncertain; ground-truth labels are used only for reward construction, not at inference. Trained with GRPO on Qwen3.5-2B, SVR achieves competitive accuracy on seven math reasoning benchmarks using an average of only 2.99 inference turns, outperforming fixed-budget and oracle-guided baselines. The work advances the line of research on efficient, oracle-free test-time compute scaling.
RLAES: Reinforcement learning framework for joint essay scoring and feedback generation with rubric rewards
Researchers propose RLAES, a unified LLM framework that jointly optimizes automated essay scoring (AES) and feedback generation via reinforcement learning. The system introduces Rubric-based Feedback Evaluation (RFE) with 166 fine-grained binary rubric items and an LLM-as-judge, plus Adaptive Gated Feedback Optimization (AGFO) and Adjacent Contrastive Reasoning (ACR) for score calibration. On the ASAP benchmark, RLAES-AGFO achieves QWK=0.803 among LLM-based methods while maintaining feedback quality comparable to GPT-5.5. Code and datasets are publicly released.
GR2: Generative Reasoning Re-Ranker applies RL with verifiable rewards to industrial-scale LLM recommendation
GR2 (Generative Reasoning Re-Ranker) is a new framework that applies reinforcement learning with verifiable rewards to the re-ranking stage of industrial recommendation systems, a step largely overlooked by prior LLM-based recommendation research. The system combines semantic ID mid-training, reasoning-trace distillation from a stronger teacher model, and purpose-built RL rewards, plus a context compressor and On-Policy Distillation to make it viable at scale. Deployed on industrial traffic, GR2 achieves +18.7% R@1 and +9.6% N@3 over legacy baselines. The paper also identifies a critical reward-hacking failure mode where LLMs exploit position bias or preserve input order, motivating conditional verifiable rewards.
CORA: Consistency-Oriented Reasoning Alignment addresses thinking-answer gap in multimodal RLVR
Researchers identify and analyze a systematic inconsistency between reasoning traces and final answers in RLVR-trained large vision-language models, showing the problem persists throughout GRPO training and inference. They propose CORA, which introduces a lightweight plug-and-play consistency reward model and a Hybrid Reward Advantage Splitting (HRAS) mechanism to coordinate task and consistency optimization. Experiments across multimodal reasoning benchmarks show CORA improves both task performance and reasoning faithfulness.
QUBRIC: Co-designing queries and rubrics for RL beyond verifiable rewards
QUBRIC is a framework that jointly optimizes queries and rubrics for reinforcement learning in settings where rewards are not strictly verifiable. The approach uses teacher-derived key points to rewrite open-ended queries into evaluable scenarios, applies contrastive rubric generation to capture teacher-policy gaps, and filters for learnability before GRPO training. Trained only on instruction-following data, QUBRIC achieves a +5.5 point gain on ArenaHard over an SFT baseline and transfers to legal, moral, and narrative reasoning benchmarks (+6.3 points average), suggesting rubric-based RL can complement RLVR in non-verifiable domains.
Gazer: Training-free semantic correction for autoregressive visual models using MLLM feedback
Researchers introduce Gazer, a training-free framework that integrates multimodal large language model feedback into the sampling loop of autoregressive visual models (AVMs) to correct semantic errors during generation. The system operates in two stages: Reflective Diagnosis identifies semantic errors in intermediate generation states, and Semantic Correction rewinds and adjusts the generation trajectory to better match the target prompt. Experiments on compositional image and video benchmarks show improved semantic alignment and compositional accuracy across multiple AVMs without additional training. The work addresses a known weakness of next-scale prediction AVMs, where semantic errors accumulate across discrete generation scales.