6arXiv cs.LG (Machine Learning)·
TerraZero: Procedural driving simulator achieves top InterPlan benchmark with zero demonstrations
TerraZero is a procedural driving simulator and self-play training stack that trains autonomous driving agents entirely from reinforcement learning with no human demonstrations, achieving 1.3M agent-steps per second on a single server-grade GPU. The system uses real-world map geometry but populates scenarios with randomized agents and dynamics, enabling unbounded scenario diversity. TerraZero is the first fully learned policy to top the InterPlan long-tail benchmark and posts best-in-class collision and time-to-collision scores on val14, while also being competitive on Waymo Open Sim Agents realism metrics. The result is notable as a demonstration that zero-demonstration RL self-play can match or exceed demonstration-anchored methods on standard autonomous driving benchmarks.
Related guides (2)
Related events (8)
Learning Montezuma's Revenge from a Single Demonstration
OpenAI trained a reinforcement learning agent to achieve a score of 74,500 on Montezuma's Revenge using a single human demonstration, surpassing all previously published results. The method is straightforward: the agent plays episodes starting from carefully selected states drawn from the demonstration, optimizing game score via PPO. This approach demonstrates that imitation-seeded curriculum learning can dramatically improve exploration in hard-exploration environments. The same PPO algorithm underpins OpenAI Five.
Scaling Up Reinforcement Learning for Traffic Smoothing: A 100-AV Highway Deployment
Berkeley AI Research (BAIR) deployed 100 RL-controlled autonomous vehicles into real rush-hour highway traffic on Interstate 24 near Nashville to dampen stop-and-go waves and reduce fuel consumption. The RL controllers were trained in data-driven simulations built from real highway trajectory data, using only local sensor inputs (speed, lead vehicle speed, gap) to enable decentralized deployment on standard vehicles. Reward design balanced wave smoothing, energy efficiency, safety, comfort, and adherence to human driving norms. The paper documents the sim-to-real transfer challenges encountered during this large-scale field experiment.
More on Dota 2: OpenAI Self-Play Reaches Superhuman Performance
OpenAI reports that a self-play reinforcement learning system progressed from below high-ranked human level to beating top professional Dota 2 players within one month, using only 1v1 mid-lane play. The post highlights self-play as a mechanism that automatically improves training data quality as the agent improves, contrasting it with supervised learning's dependence on fixed datasets. The result is presented as evidence that sufficient compute combined with self-play can rapidly close and exceed human-level performance gaps.
SpatialWorld benchmark evaluates interactive spatial reasoning of multimodal agents in real-world tasks
Researchers introduce SpatialWorld, a benchmark for evaluating interactive spatial understanding of multimodal agents across 760 human-annotated tasks spanning household, travel, and social domains. The benchmark integrates eight simulation backends under a shared protocol, requiring agents to operate under vision-only partial observability with egocentric inputs. Evaluation of 15 agents reveals that even the strongest model, GPT-5, achieves only 17.4% task success rate, exposing significant gaps in active exploration and long-horizon planning. The work highlights a mismatch between task success and execution efficiency as a key bottleneck for spatial agents.
CARLA-GS: Modular pipeline for photorealistic autonomous driving corner-case synthesis using LLMs and Gaussian splatting
CARLA-GS is a new modular framework for synthesizing safety-critical corner cases for autonomous driving evaluation, decoupling visual representation (editable Gaussian scenes), semantic reasoning (multi-agent LLM for waypoint generation), and physics simulation (CARLA with PID control). The pipeline reconstructs real driving scenes from the Waymo Open Dataset, uses LLMs to identify risky interactions and generate intent-level trajectories, then re-projects simulated states into Gaussian scenes for photorealistic ego-centric rendering. The approach addresses a key gap in AV safety evaluation: generating spatiotemporally consistent, physically feasible, and semantically meaningful rare scenarios at scale.
OpenAI Releases Procgen Benchmark for RL Generalization
OpenAI released Procgen Benchmark, a suite of 16 procedurally-generated environments designed to measure how quickly reinforcement learning agents learn generalizable skills. The benchmark targets a core challenge in RL: distinguishing memorization of specific environments from genuine skill generalization. Its procedural generation ensures agents cannot overfit to fixed level layouts.
First multiplayer world model for dynamic environments using latent diffusion on Rocket League gameplay
Researchers introduce the first multiplayer world model capable of conditioning on action streams from multiple agents simultaneously, trained on 10,000 hours of Rocket League gameplay. The 5-billion-parameter latent diffusion model generates four-player matches in real time at 20 FPS on a single Nvidia B200 GPU, with rollouts remaining stable far beyond the training horizon (tested to five minutes, observed for hours). The paper systematically investigates codec design, generative objectives, and multiplayer conditioning, and releases the dataset, codebase, and a live demo. The work advances world model research by addressing multi-agent attribution and coherence under complex physical dynamics.
Generalizing from Simulation: OpenAI Sim-to-Real Robotics Transfer
OpenAI published results on sim-to-real transfer for robot controllers, demonstrating that policies trained entirely in simulation can be deployed on physical robots and respond to unplanned environmental changes. The work represents a shift from open-loop to closed-loop control systems in robotics. This is a 2017 research milestone predating current frontier model work but relevant to the historical trajectory of OpenAI's robotics program.