What it is
Reinforcement Learning with Verifiable Rewards (RLVR) is a post-training technique that fine-tunes a language model using reward signals derived from automatic verification rather than human preference judgements. The canonical setup: the model generates a candidate answer, an external checker (a math evaluator, a code executor, a rubric scorer) determines whether it is correct, and a policy gradient algorithm — most commonly GRPO — updates the model weights to increase the probability of correct outputs. The verifiability constraint is the defining feature: it limits the technique to domains where correctness can be checked programmatically, but in exchange it delivers a reward signal that is cheap, scalable, and free of annotator bias.
How it works
The training loop has three moving parts:
1. Rollout generation. The current policy samples multiple completions for each prompt. In on-policy distillation variants, a teacher model may also contribute rollouts. 2. Reward assignment. Each completion is scored — pass/fail for math answers or code execution; rubric criteria scores for open-ended tasks; richer signals (execution traces, expert corrections) in extended variants. 3. Policy update. A policy gradient step (e.g. GRPO) increases the log-probability of high-reward completions relative to low-reward ones, typically with a KL penalty to prevent the policy from drifting too far from the reference model.
The simplicity of this loop is both its strength and the source of its known failure modes.
Why it matters
RLVR is the primary mechanism behind the reasoning capability gains seen in recent frontier and open-weight models. It requires no human labellers at scale, generalises across domains wherever a verifier exists, and produces measurable, benchmark-trackable improvements. The technique has moved from a research curiosity to a standard component of post-training pipelines, with applications now spanning mathematical reasoning, code generation, scientific QA, long-context retrieval, and commercial agent tasks.
Active failure modes and the research frontier
The current wave of RLVR research is largely a systematic attack on the technique's known weaknesses:
Token credit misassignment
Policy gradient updates treat all tokens in a completion roughly equally, but high-frequency formatting tokens (punctuation, structural markers) appear in both correct and incorrect outputs and contribute large but uninformative gradient mass. DelTA addresses this with discriminative per-token coefficients that amplify gradient directions specific to correct vs. incorrect completions and downweight shared tokens — yielding +3.26 and +2.62 average points on Qwen3-8B-Base and Qwen3-14B-Base respectively across seven math benchmarks.
Blunt rubric aggregation
When rubric-based rewards are used for open-ended tasks, static criterion weights conflate human-assigned importance with current optimization utility. Many criteria are already saturated (the model always satisfies them) or unreachable (it never does), making their gradient signal useless. POW3R dynamically reweights criteria during training using rollout-level contrast — emphasising criteria that currently differentiate policy outputs — and reaches equivalent performance in 2.5–4× fewer training steps than vanilla GRPO with static rubric rewards.
Rollout compute cost
Generating full rollouts is the dominant wall-clock cost in RLVR training. Two complementary strategies attack this: TOPD (Truncated On-Policy Distillation) matches full rollout performance using only 10% of the rollout horizon; POPD (Progressive OPD) gradually expands the horizon during training, achieving up to 3× training efficiency improvement on mathematical reasoning.
The single-bit reward ceiling
Binary pass/fail rewards discard information present in execution traces, partial solutions, and expert corrections. DistIL replaces the single-bit signal with a distributional imitation learning objective over rich feedback, using a forward cross-entropy loss that provides monotonic policy improvement guarantees — a property not shared by reverse KL or Jensen-Shannon objectives used in prior self-distillation work. It outperforms RLVR and self-distillation baselines on scientific reasoning, coding, and hard math.
Training trajectory inefficiency
RELEX takes a different angle: rather than improving the reward or the update rule, it observes that RLVR weight update trajectories are near-rank-1 and near-linearly predictable. A rank-1 approximation captures most downstream performance gains, and the rank-1 projection also acts as a denoising filter that discards stochastic optimization noise. RELEX observes a short training window, estimates the rank-1 subspace, and extrapolates future checkpoints via linear regression — no additional training required. On Qwen2.5-Math-1.5B and Qwen3 models, it matches or exceeds full RLVR performance at 15% of training steps and can extrapolate 10–20× beyond the observed prefix.
Unsupervised and reward-hacking regimes
When external ground truth is unavailable, single-reward RLIF approaches suffer from reward hacking and entropy collapse. The multi-reward RLIF framework decomposes the training signal into an answer-level reward (cluster voting) and a completion-level reward (token-wise self-certainty), stabilised with GDPO-based normalization and KL-Cov regularization targeting low-entropy token distributions. The result approaches supervised RLVR performance without requiring external labels.
Domain expansion
RLVR's reach is widening beyond its math/code origins. LongTraceRL constructs training data from knowledge-graph random walks with tiered distractors derived from search agent trajectories, and applies rubric rewards with entity-level process supervision — but only on correct responses, to prevent reward hacking. This extends RLVR to long-context multi-hop reasoning, with consistent gains across five benchmarks on models from 4B to 30B parameters. At the commercial end, Ecom-RLVE applies the paradigm to e-commerce conversational agents via adaptive verifiable environments, demonstrating that wherever a domain can be instrumented to verify agent actions, RLVR-style training becomes applicable.
Tradeoffs and when not to use it
RLVR is the right tool when: (a) a reliable automatic verifier exists for the target task, (b) the task has a meaningful difficulty gradient so the model can learn from failures, and (c) compute for rollout generation is available. It is the wrong tool when the target domain lacks a verifier (open-ended creative tasks, nuanced judgment), when the model is already near ceiling on the verifiable metric, or when rollout cost is prohibitive and neither truncation nor trajectory extrapolation closes the gap. In those cases, supervised fine-tuning, RLHF, or self-distillation remain the practical alternatives.
