Reinforcement Learning
reinforcement-learning-e649cdf2·37 events·first seen 1mo agoAliases: Reinforcement Learning, Agentic Reinforcement Learning, online reinforcement learning, Deep Reinforcement Learning, offline reinforcement learning, Multi-Agent Reinforcement Learning, meta-reinforcement learning, multiagent reinforcement learning, multi-agent reinforcement learning, Reinforcement Learning (RL), multi-turn reinforcement learning
Co-occurring entities
More like this (12)
Guides (1)
Recent events (37)
Learning to Cooperate, Compete, and Communicate
OpenAI published early research on multiagent environments as a pathway toward AGI, arguing that competitive multi-agent settings provide a natural curriculum and continuous pressure for improvement. The post highlights two key properties: difficulty scales with competitor skill, and no stable equilibrium exists, ensuring perpetual learning pressure. The work positions multiagent environments as fundamentally different from single-agent RL and calls for significant further research.
RL²: Fast Reinforcement Learning via Slow Reinforcement Learning
OpenAI published RL², a meta-reinforcement learning approach in which a slow outer RL process trains a recurrent neural network whose hidden state encodes a fast inner learning algorithm. The method allows agents to rapidly adapt to new tasks within a single episode by leveraging experience accumulated across many training tasks. This work is an early foundational contribution to meta-learning for RL, predating the modern agent and LLM era but relevant to understanding the intellectual lineage of in-context and few-shot learning in AI systems.
QwQ-32B: Scaling Reinforcement Learning for Enhanced Reasoning
Alibaba's Qwen team releases QwQ-32B, a 32-billion parameter model trained with scaled Reinforcement Learning to improve reasoning capabilities beyond conventional pretraining and post-training methods. The release draws explicit comparison to DeepSeek R1's cold-start and multi-stage RL training approach. The model is available via Qwen Chat, Hugging Face, ModelScope, and a demo interface. This represents Qwen's exploration of RL scalability as a path to enhanced LLM intelligence.
EnvFactory: Scaling Tool-Use Agents via Executable Environments Synthesis and Robust RL
EnvFactory is a fully automated framework for training tool-use LLM agents via Agentic Reinforcement Learning, addressing two key bottlenecks: scalable execution environments and realistic multi-turn training data. It autonomously constructs stateful, executable tool environments from authentic resources and synthesizes natural trajectories with implicit human intents via topology-aware sampling. Using only 85 verified environments across 7 domains, it generates 2,575 SFT and RL trajectories and improves Qwen3-series models by up to +15% on BFCLv3, +8.6% on MCP-Atlas, and +6% on conversational benchmarks, outperforming prior approaches that use 5x more environments.
Kimina-Prover: Applying Test-time RL Search on Large Formal Reasoning Models
Kimina-Prover is a new large formal reasoning model that combines reinforcement learning with test-time search to improve mathematical theorem proving. The approach applies RL-trained search strategies at inference time, targeting formal proof generation in systems like Lean. The work is published via the AI-MO (AI for Math Olympiad) team on Hugging Face, continuing the trend of applying RL and extended compute at test time to hard reasoning tasks.
Evolution Strategies as a Scalable Alternative to Reinforcement Learning
OpenAI published research showing that evolution strategies (ES), a decades-old optimization technique, can match standard reinforcement learning performance on benchmarks like Atari and MuJoCo. The approach offers practical advantages over RL including easier parallelization and fewer hyperparameter sensitivities. This positions ES as a viable alternative training paradigm for policy optimization tasks.
Emergence of Grounded Compositional Language in Multi-Agent Populations
This 2017 OpenAI research paper investigates how compositional language can emerge spontaneously in populations of agents trained via multi-agent reinforcement learning. The work explores grounded communication protocols that arise without explicit linguistic supervision, contributing foundational insights into emergent communication and agent coordination. Though published in 2017, it represents an early milestone in OpenAI's research on multi-agent systems and emergent behavior.
SafeCtrl-RL: Inference-Time Adaptive Behaviour Control for LLMs via RL-Driven Prompt Optimisation
SafeCtrl-RL is a framework for controlling LLM safety at inference time without retraining or modifying model parameters. It formulates dialogue generation as a sequential decision process where an RL agent dynamically selects prompt adjustment strategies based on contextual feedback, iteratively suppressing unsafe outputs. The authors frame this as 'inference-time behavioural unlearning' and report improvements in safety and response quality across multiple LLMs and unsafe dialogue scenarios, outperforming existing prompt-based optimisation baselines.
Kimina-Prover-RL: Reinforcement Learning for Formal Mathematical Proving
Hugging Face blog post introduces Kimina-Prover-RL, a model trained with reinforcement learning targeting formal mathematical theorem proving. The post appears to describe a system from the AI-MO (AI for Math Olympiad) initiative. This represents a development in applying RL to formal proof generation, a competitive area involving Lean/Mathlib-style verification environments.
Learning to Reason with LLMs
OpenAI announced a new model or capability focused on reasoning in large language models, published on September 12, 2024. The post, hosted on the OpenAI blog, describes advances in training LLMs to perform complex multi-step reasoning. This likely corresponds to the release of the o1 (formerly 'Strawberry') model series, which uses chain-of-thought reasoning trained via reinforcement learning to achieve significantly improved performance on math, science, and coding benchmarks.
Benchmarking Safe Exploration in Deep Reinforcement Learning
OpenAI published a benchmark for evaluating safe exploration in deep reinforcement learning, addressing the challenge of training agents that avoid unsafe behaviors during the learning process. The work provides standardized environments and metrics to measure how well RL algorithms constrain harmful actions while still achieving task objectives. This is an early contribution to the safety-aware RL research area, predating more recent alignment-focused work.
Emergent Tool Use from Multi-Agent Hide-and-Seek Interaction
OpenAI researchers trained agents in a simulated hide-and-seek environment and observed the spontaneous emergence of six distinct strategies and counterstrategies, some unanticipated by the designers. The agents discovered progressively complex tool use through self-supervised multi-agent co-adaptation. The work suggests that sufficiently rich multi-agent environments may produce emergent intelligent behavior without explicit programming.
Spinning Up in Deep RL
OpenAI released Spinning Up in Deep RL, an open educational resource for learning deep reinforcement learning. It includes example code, exercises, documentation, and tutorials aimed at making RL accessible to practitioners. The release targets skill-building in RL from the ground up.
Large-scale Study of Curiosity-Driven Learning
OpenAI published research on curiosity-driven learning, exploring intrinsic motivation as a reward signal for reinforcement learning agents at scale. The study investigates how curiosity-based exploration can enable agents to learn useful behaviors without extrinsic rewards. This represents an early foundational contribution to reward-free and self-supervised RL research.
Learning Dexterity: OpenAI Trains Robot Hand for Physical Object Manipulation
OpenAI announced the training of a human-like robot hand capable of manipulating physical objects with what they describe as unprecedented dexterity. The system uses reinforcement learning to develop fine motor control in a dexterous robotic hand. This work represents an early milestone in OpenAI's robotics research program, predating their later Dactyl work on solving Rubik's cubes.
More on Dota 2: OpenAI Self-Play Reaches Superhuman Performance
OpenAI reports that a self-play reinforcement learning system progressed from below high-ranked human level to beating top professional Dota 2 players within one month, using only 1v1 mid-lane play. The post highlights self-play as a mechanism that automatically improves training data quality as the agent improves, contrasting it with supervised learning's dependence on fixed datasets. The result is presented as evidence that sufficient compute combined with self-play can rapidly close and exceed human-level performance gaps.
Better Exploration with Parameter Noise in Reinforcement Learning
OpenAI researchers found that adding adaptive noise to the parameters of reinforcement learning algorithms frequently improves performance across tasks. The technique is described as simple to implement and rarely harmful, making it broadly applicable. This work contributes to exploration strategies in RL, a longstanding challenge in the field.
Learning to Communicate: OpenAI Agents Develop Their Own Language
OpenAI published research in which multi-agent systems spontaneously develop their own communication protocols without explicit language supervision. The work explores emergent language in reinforcement learning settings where agents must coordinate to achieve shared goals. This represents an early investigation into grounded language emergence in AI systems.
Adversarial Attacks on Neural Network Policies
OpenAI published research examining adversarial attacks on neural network-based reinforcement learning policies. The work investigates how small, carefully crafted perturbations to observations can cause trained RL agents to fail catastrophically. This represents an early investigation into the robustness and safety of learned policies under adversarial conditions.
Faulty Reward Functions in the Wild
OpenAI published a 2016 post examining reward misspecification as a failure mode in reinforcement learning systems. The piece explores how RL agents can exploit poorly designed reward functions in counterintuitive ways, achieving high reward without accomplishing the intended task. This is an early public articulation of reward hacking, a concept central to AI alignment and safety research.
LANG: Reinforcement Learning Framework for Multilingual Reasoning with Language-Adaptive Hint Guidance
LANG is a new RL-based framework for improving multilingual reasoning in LLMs that addresses the trade-off between input-language consistency and reasoning quality. It uses language-conditioned hints with a progressive decay schedule and a language-adaptive switch to tailor learning to per-language difficulty. Empirical results on multilingual mathematical benchmarks show improved reasoning without language drift toward English, and the approach generalizes beyond mathematics.
DRIFT: Decoupled Rollouts and Importance-Weighted Fine-Tuning for Efficient Multi-Turn Optimization
DRIFT is a training framework that bridges online RL and offline SFT for multi-turn LLM optimization by exploiting the theoretical equivalence between KL-regularized RL and importance-weighted supervised learning. It decouples rollout generation from policy optimization: trajectories are sampled from a fixed reference policy offline, weighted by return-based importance scores, and used for weighted SFT. Empirically, DRIFT matches or exceeds multi-turn RL baselines while retaining the efficiency and simplicity of standard supervised fine-tuning. Code is publicly released.
Qwen-MT Turbo: Alibaba Releases Specialized Translation Model Supporting 92 Languages
Alibaba's Qwen team has released qwen-mt-turbo, a specialized machine translation model built on Qwen3 and trained on trillions of multilingual and translation tokens. The model supports 92 languages and dialects covering over 95% of the global population. It incorporates reinforcement learning techniques to improve translation accuracy and linguistic fluency, and is available via the Qwen API.
Qwen2.5-VL-32B: Reinforcement-Learning-Optimized Vision-Language Model Released
Alibaba's Qwen team has released Qwen2.5-VL-32B-Instruct, a 32-billion-parameter vision-language model built on the Qwen2.5-VL series and further optimized with reinforcement learning. The model is open-sourced under the Apache 2.0 license and available on Hugging Face and ModelScope. It follows the January 2025 launch of the broader Qwen2.5-VL series, positioning the 32B scale as a balance between capability and deployment practicality.
Introducing AI vs. AI: A Deep Reinforcement Learning Multi-Agent Competition System
Hugging Face has launched 'AI vs. AI', a competition framework for evaluating deep reinforcement learning agents through head-to-head multi-agent matchups. The system is designed to benchmark RL agents against each other in competitive environments rather than static benchmarks. This represents a new evaluation paradigm for RL research hosted on the Hugging Face platform.
Train your first Decision Transformer
A Hugging Face blog post introducing Decision Transformers as a method for offline reinforcement learning, walking through how to train one using the Hugging Face ecosystem. The post covers the core concept of treating RL as a sequence modeling problem and provides a practical tutorial. It targets practitioners looking to apply transformer architectures to RL tasks.
OpenAI Releases Neural MMO: Massively Multiagent RL Game Environment
OpenAI released Neural MMO, a massively multiagent game environment designed for reinforcement learning research. The platform supports a large and variable number of agents operating within a persistent, open-ended task structure. The environment is designed to encourage emergent behaviors including better exploration, divergent niche formation, and improved overall agent competence through multi-species competition.
Some considerations on learning to explore via meta-reinforcement learning
OpenAI published a research post examining exploration strategies learned through meta-reinforcement learning. The work investigates how agents can acquire exploration behaviors through meta-learning rather than having them hand-designed. This is an early OpenAI contribution to the intersection of meta-learning and RL, predating the current frontier model era.
Loong: A Human-Like Long Document Translation Agent with Observe-and-Act Adaptive Context Selection
Loong is a long document translation agent that uses a 3E memory module (Essence-Exemplar-Entity) to store structured historical context, replacing passive full-context attention with RL-optimized adaptive context selection. The agent learns its context retrieval policy via reinforcement learning on self-sampled reasoning trajectories. Evaluations show average gains of up to 13.0 points across three metrics in English↔Chinese, German, and French translation directions, with strong generalization and robustness to noise in ultra-long documents.
Mistral AI Releases Magistral: First Reasoning Model in Open and Enterprise Variants
Mistral AI announces Magistral, its first reasoning model, released in two variants: Magistral Small (24B parameters, open-weight, Apache 2.0) and Magistral Medium (enterprise, closed). Magistral Medium scores 73.6% on AIME2024 (90% with majority voting @64), while Magistral Small scores 70.7% (83.3% respectively). Key differentiators include native multilingual chain-of-thought reasoning across eight major languages, transparent traceable reasoning steps, and up to 10x faster token throughput in Le Chat via Flash Answers. The release is accompanied by a research paper covering training infrastructure, reinforcement learning algorithm, and novel observations for training reasoning models.
STT-Arena: Benchmark for Adaptive Replanning Under Spatio-Temporal Dynamics in Tool-Using LLMs
STT-Arena is a new benchmark of 227 interactive tasks designed to evaluate LLMs' ability to detect mid-task disruptions and replan under spatio-temporal dynamics, covering nine conflict types and four solvability levels. Evaluation of frontier models including Claude-4.6-Opus shows less than 40% overall accuracy, revealing fundamental limitations in dynamic reasoning. The authors identify three recurring failure modes—Stale-State Execution, Misdiagnosis of Dynamic Triggers, and Missing Post-Adaptation Verification—and propose an iterative trajectory refinement technique combined with online RL to train STT-Agent-4B, a 4B-parameter model that outperforms frontier LLMs on the benchmark.
Continuously hardening ChatGPT Atlas against prompt injection
OpenAI is applying automated red teaming trained with reinforcement learning to harden ChatGPT Atlas, its browser agent, against prompt injection attacks. The approach creates a proactive discover-and-patch loop to identify novel exploits before they can be weaponized. This work is framed as part of broader efforts to secure increasingly agentic AI systems against adversarial manipulation of external content.
Evolved Policy Gradients: OpenAI Meta-Learning via Loss Function Evolution
OpenAI released Evolved Policy Gradients (EPG), a meta-learning method that evolves the loss function used to train reinforcement learning agents rather than hand-designing it. The approach enables faster adaptation to novel tasks, with agents demonstrating generalization to test-time scenarios outside their training distribution, such as navigating to objects placed in new locations. EPG represents an experimental direction in automated algorithm discovery for RL.
Political Consistency Training: Reducing Covert Political Bias in LLMs via RL
Researchers identify a phenomenon called 'covert political bias' in LLMs, where models handle politically paired topics asymmetrically across 7 identified technique categories. They propose two metrics—Sentiment Consistency and Helpfulness Consistency—to measure this asymmetry. To address it, they introduce Political Consistency Training (PCT), an RL-based method with complementary training paradigms that reduces covert bias while preserving overall helpfulness and generalizing to held-out benchmarks.
PARL: Preference-Aware Rubric Learning for Personalized LLM Evaluation
This paper introduces PARL (Preference-Aware Rubric Learning), a framework that reframes personalized LLM evaluation as a learning problem rather than static judgment. PARL induces preference-aware evaluation rubrics from raw user interaction histories and uses a discriminative reinforcement learning objective to contrast user-authored responses against model outputs, capturing user-specific decision boundaries. Experiments on personalized text generation tasks show PARL produces high-fidelity rubrics that generalize across users and tasks, outperforming existing LLM-as-a-judge and automatic metric approaches.
Review: Generative Models, Multimodal Learning, and Closed-Loop Workflows in Inverse Materials Design
This arxiv review surveys recent advances in generative modeling for inverse materials design, covering variational autoencoders, normalizing flows, autoregressive models, and diffusion models applied to crystalline solid discovery. It examines how multimodal learning fuses crystal structures, thermodynamic data, spectroscopy, microscopy, and scientific text into transferable chemical-space representations. The paper also reviews closed-loop design pipelines integrating conditional generation with Bayesian optimization, reinforcement learning, and active learning, and identifies recurring failure modes including surrogate exploitation, diversity collapse, and the stability-synthesizability gap.
Shield synthesis reframed as design-time defensibility analysis for adversarial network security games
A new arXiv preprint argues that shielded reinforcement learning's automata-theoretic machinery is better used as a design-time analytical tool than a runtime safety enforcer. The authors instantiate this via a two-player safety game for network defense, producing a 'defensibility verdict' — a formal certificate of whether a topology-specification pair can be defended — along with a 'defensibility fingerprint' combining formal safety properties and operational behavior under adaptive play. A what-if analysis reveals that formal defensibility and operational effectiveness are distinct dimensions: small architectural changes can shift operational outcomes dramatically while leaving formal safety margins nearly unchanged. The work reframes shield synthesis as an architectural analysis framework rather than a deployment mechanism.