6arXiv cs.AI (Artificial Intelligence)·
Pictura: GPU-accelerated simulator enables first large-scale driving self-play from egocentric camera images
Researchers introduce Pictura, a GPU-accelerated multi-agent driving simulator that renders each agent's egocentric camera view at every training step, enabling self-play without privileged observations like exact poses or velocities. Using Pictura, they train Alberti via PPO over 50B agent steps (~35M km of driving), the first large-scale driving policy trained directly from perspective images. Alberti approaches the performance of privileged vectorized counterparts and zero-shot transfers to Waymo Open Motion Dataset layouts, outperforming privileged agents there. The work addresses a longstanding representation gap between simulation-trained policies and real deployed agents.
Related guides (3)
Related events (8)
TerraZero: Procedural driving simulator achieves top InterPlan benchmark with zero demonstrations
TerraZero is a procedural driving simulator and self-play training stack that trains autonomous driving agents entirely from reinforcement learning with no human demonstrations, achieving 1.3M agent-steps per second on a single server-grade GPU. The system uses real-world map geometry but populates scenarios with randomized agents and dynamics, enabling unbounded scenario diversity. TerraZero is the first fully learned policy to top the InterPlan long-tail benchmark and posts best-in-class collision and time-to-collision scores on val14, while also being competitive on Waymo Open Sim Agents realism metrics. The result is notable as a demonstration that zero-demonstration RL self-play can match or exceed demonstration-anchored methods on standard autonomous driving benchmarks.
CARLA-GS: Modular pipeline for photorealistic autonomous driving corner-case synthesis using LLMs and Gaussian splatting
CARLA-GS is a new modular framework for synthesizing safety-critical corner cases for autonomous driving evaluation, decoupling visual representation (editable Gaussian scenes), semantic reasoning (multi-agent LLM for waypoint generation), and physics simulation (CARLA with PID control). The pipeline reconstructs real driving scenes from the Waymo Open Dataset, uses LLMs to identify risky interactions and generate intent-level trajectories, then re-projects simulated states into Gaussian scenes for photorealistic ego-centric rendering. The approach addresses a key gap in AV safety evaluation: generating spatiotemporally consistent, physically feasible, and semantically meaningful rare scenarios at scale.
GaP: Graph-as-Policy multi-agent harness for reliable robot automation via self-learning computation graphs
Researchers introduce Graph-as-Policy (GaP), a multi-agent coding harness that generates directed computation graphs combining perception, planning, and control nodes for robotic 'Variational Automation' tasks — those with high variation in object geometry and pose. GaP uses an internal simulation environment to rehearse and iteratively refine graph structures in parallel, improving success rates without relying solely on model-free policies. Evaluation across 8 new benchmarks (4 simulated, 4 real-world) shows significant outperformance over baselines. The work bridges agentic coding systems with interpretable robot programming and Task and Motion Planning.
OmniAgent: POMDP-based active perception agent for long video understanding with test-time scaling
Researchers introduce OmniAgent, a multimodal agent that reformulates long video understanding as a POMDP-based iterative Observation-Thought-Action cycle, selectively distilling audio-visual cues into persistent textual memory rather than processing all frames uniformly. The system uses Agentic Supervised Fine-Tuning and a novel reinforcement learning method (TAURA) with turn-level entropy for credit assignment. OmniAgent demonstrates positive test-time scaling and achieves state-of-the-art open-source results across ten benchmarks, with its 7B model outperforming Qwen2.5-VL-72B on LVBench (50.5% vs. 47.3%).
Project Genie Expands with Street View Integration for Real-World Simulation
DeepMind is expanding Project Genie access to Google AI Ultra subscribers globally and introducing a new capability that uses Street View data to simulate real-world places. Project Genie, previously known for generating interactive 2D environments from images, is now incorporating real-world geographic imagery as a conditioning source. The announcement signals a move toward grounding generative world models in actual physical environments.
Nvidia and Hugging Face overview the state of simulation for Physical AI
A joint Nvidia/Hugging Face blog post surveys the current landscape of simulation tools and techniques for Physical AI — AI systems that interact with the physical world, such as robotics and autonomous systems. The piece covers the role of simulation in training and evaluating embodied agents, likely addressing sim-to-real transfer, synthetic data generation, and key platforms. As a tier-2 commentary from a major hardware and ML infrastructure player, it reflects growing industry attention to simulation as a bottleneck for physical AI development.
GS-Agent: Multi-agent framework for generating physically realistic 4D worlds from natural language
GS-Agent is an end-to-end multi-agent system that generates dynamic, physically plausible 4D worlds from natural language descriptions by integrating physics engines in the loop. The framework decomposes world creation into entity management (3D asset curation, material tuning, placement, motion control) and rendering configuration, with specialized agents collaborating via code and multimodal feedback. Results demonstrate generation of scenes with rich interactions among liquids, deformable objects, and rigid bodies, with cinematic camera and lighting control. The work positions itself as a foundation for physical AI and automated creative content generation.
MobileGym: Verifiable Parallel Simulation Platform for Mobile GUI Agent Training
MobileGym is a browser-hosted simulation environment for mobile GUI agent research that enables deterministic outcome verification via structured JSON state and scalable online RL through hundreds of parallel instances (~400 MB/instance, ~3s cold start). The accompanying MobileGym-Bench provides 416 parameterized task templates across 28 apps with deterministic judges. A sim-to-real case study using GRPO on Qwen3-VL-4B-Instruct achieves +12.8 percentage points on the 256-task test set, with real-device execution retaining 95.1% of simulation-side training gains.