Solving Rubik's Cube with a Robot Hand via Reinforcement Learning and Automatic Domain Randomization
OpenAI trained neural networks to solve a Rubik's Cube using a dexterous robot hand, with training conducted entirely in simulation via reinforcement learning. A new technique called Automatic Domain Randomization (ADR) enables the system to generalize to real-world physical perturbations not seen during training. The work demonstrates that sim-to-real transfer can achieve unprecedented dexterity in manipulation tasks.
Related guides (3)
Related events (8)
Learning Dexterity: OpenAI Trains Robot Hand for Physical Object Manipulation
OpenAI announced the training of a human-like robot hand capable of manipulating physical objects with what they describe as unprecedented dexterity. The system uses reinforcement learning to develop fine motor control in a dexterous robotic hand. This work represents an early milestone in OpenAI's robotics research program, predating their later Dactyl work on solving Rubik's cubes.
Sim-to-real transfer of robotic control with dynamics randomization
OpenAI published research on transferring robotic control policies trained in simulation to real-world robots using dynamics randomization. The technique involves varying physical parameters during simulation training so that the real world appears as just another variation, enabling zero-shot sim-to-real transfer. This was an early foundational contribution to the sim-to-real robotics research thread.
Learning Montezuma's Revenge from a Single Demonstration
OpenAI trained a reinforcement learning agent to achieve a score of 74,500 on Montezuma's Revenge using a single human demonstration, surpassing all previously published results. The method is straightforward: the agent plays episodes starting from carefully selected states drawn from the demonstration, optimizing game score via PPO. This approach demonstrates that imitation-seeded curriculum learning can dramatically improve exploration in hard-exploration environments. The same PPO algorithm underpins OpenAI Five.
Generalizing from Simulation: OpenAI Sim-to-Real Robotics Transfer
OpenAI published results on sim-to-real transfer for robot controllers, demonstrating that policies trained entirely in simulation can be deployed on physical robots and respond to unplanned environmental changes. The work represents a shift from open-loop to closed-loop control systems in robotics. This is a 2017 research milestone predating current frontier model work but relevant to the historical trajectory of OpenAI's robotics program.
OpenAI Five Defeats Amateur Human Teams at Dota 2
OpenAI announced that OpenAI Five, a team of five neural networks trained via self-play, has begun defeating amateur human teams at Dota 2. This represented an early milestone in applying reinforcement learning to complex, long-horizon multi-agent environments. The system was trained using large-scale distributed RL, demonstrating that neural networks could coordinate in real-time strategy games without hand-crafted rules.
Reinforcement Learning with Prediction-Based Rewards (Random Network Distillation)
OpenAI introduces Random Network Distillation (RND), a curiosity-driven exploration method for reinforcement learning that uses prediction error on a fixed random neural network as an intrinsic reward signal. RND is the first method to exceed average human performance on Montezuma's Revenge, a notoriously hard-exploration Atari game. The approach is simple to implement and compatible with standard RL algorithms, offering a scalable alternative to count-based or dynamics-model exploration bonuses.
Dota 2 with Large Scale Deep Reinforcement Learning
OpenAI published a detailed account of the OpenAI Five system that defeated world-champion Dota 2 players using large-scale deep reinforcement learning. The work describes the training infrastructure, self-play curriculum, and scaling properties that enabled superhuman performance in a complex multi-agent environment. This represents a landmark result in applying RL at scale to long-horizon, high-dimensional tasks.
OpenAI Develops Hierarchical Reinforcement Learning Algorithm for Long-Horizon Tasks
OpenAI published research on a hierarchical reinforcement learning (HRL) algorithm that learns reusable high-level actions to solve tasks requiring thousands of timesteps. Applied to navigation problems, the algorithm discovers locomotion primitives (walking, crawling in various directions) that enable rapid mastery of new tasks. The approach addresses a core challenge in RL: efficient exploration and transfer across long-horizon tasks.