5arXiv cs.AI (Artificial Intelligence)·
CARLA-GS: Modular pipeline for photorealistic autonomous driving corner-case synthesis using LLMs and Gaussian splatting
CARLA-GS is a new modular framework for synthesizing safety-critical corner cases for autonomous driving evaluation, decoupling visual representation (editable Gaussian scenes), semantic reasoning (multi-agent LLM for waypoint generation), and physics simulation (CARLA with PID control). The pipeline reconstructs real driving scenes from the Waymo Open Dataset, uses LLMs to identify risky interactions and generate intent-level trajectories, then re-projects simulated states into Gaussian scenes for photorealistic ego-centric rendering. The approach addresses a key gap in AV safety evaluation: generating spatiotemporally consistent, physically feasible, and semantically meaningful rare scenarios at scale.
Related guides (2)
Related events (8)
Lost in Fog: Sensor Perturbations Expose Reasoning Fragility in Driving VLAs
This paper presents a controlled robustness study of Vision-Language-Action (VLA) models in autonomous driving, evaluating Alpamayo R1 (10B parameters) across ~18,000 inference trials under eight sensor perturbation types including noise, lighting extremes, and fog. The key finding is that Chain-of-Causation (CoC) reasoning consistency is a high-fidelity proxy for trajectory reliability: when CoC explanations change post-perturbation, trajectory deviation spikes 5.3× (r=0.99 across attack types). Enabling CoC generation is associated with 11.8% average improvement in trajectory accuracy, and degradation under noise is approximately linear (R²=0.957), while standard preprocessing defenses offer only marginal benefit.
VLK: Synthetic vision-language-kinematics supervision enables humanoid loco-manipulation from egocentric observations
Researchers introduce a pipeline that generates 48,000 paired vision-language-kinematics trajectories synthetically using 3D Gaussian Splatting to reconstruct indoor scenes, bypassing the need for expensive human-annotated robot data. A VLK policy trained on this data predicts whole-body kinematic trajectories from egocentric images and language instructions, which a whole-body tracker converts to physical actions. The approach is validated on a Unitree G1 humanoid performing navigation and object transport, demonstrating viable sim-to-real transfer for perception-based loco-manipulation.
CNeVA: Controllable neural variational agents for steerable traffic simulation
Researchers introduce Controllable Neural Variational Agents (CNeVA), a framework for traffic simulation that infers per-agent Gaussian behavior latents from discounted returns via conjugate variational updates, conditioning a rectified-flow trajectory generator with classifier-free guidance. The system enables interpretable steering along axes such as speed, acceleration, and safety compliance without sacrificing realism. Evaluated on the Waymo Open Motion Dataset, CNeVA achieves competitive realism while offering per-channel controllability absent from higher-ranked imitation baselines. The work also introduces soft eligibility gates to address reward signal scarcity near decision thresholds.
LabVLA: Vision-Language-Action model and RoboGenesis data engine for scientific laboratory robotics
Researchers introduce LabVLA, a Vision-Language-Action model designed to bridge written scientific protocols and physical robot execution in laboratory settings. To address the data scarcity problem, they build RoboGenesis, a simulation-based data engine that composes lab workflows from atomic skills and generates structured demonstrations across robot embodiments. LabVLA uses a two-stage training recipe combining FAST action token pretraining on a Qwen3-VL-4B-Instruct backbone with flow matching posttraining via a DiT action expert. On the LabUtopia benchmark, LabVLA achieves the highest average success rate among evaluated baselines in both in-distribution and out-of-distribution settings.
PaSBench-Video: A Streaming Video Benchmark for Proactive Safety Warning in MLLMs
PaSBench-Video is a 740-video benchmark designed to evaluate whether multimodal large language models can issue timely, accurate safety warnings during the window between a visible danger sign and an accident. Videos span four domains (driving, healthcare, daily life, industrial production) and are annotated with frame-level risk onset and accident boundaries, requiring causal temporal reasoning rather than static scene classification. Testing 13 MLLMs reveals no model exceeds 20% on the strictest metric, with recall strongly coupled to false-positive rate (Pearson r=0.64), indicating models rely on scene-level activity cues rather than genuine hazard reasoning. Performance varies sharply by domain, with driving being particularly problematic due to visual similarity between routine and hazardous scenes.
Benchmark for view-level visual evidence identification in multi-view MLLMs for autonomous driving
A new arXiv preprint introduces a multi-view visual question answering benchmark targeting evidence-source identification in autonomous driving scenarios. Given six synchronized NuScenes camera views and a question, models must identify which camera view supports the answer — not just produce a correct answer. The 122-pair benchmark spans causality, counterfactual reasoning, and intent prediction, and exposes grounding failures that answer-only evaluation misses. The work addresses a meaningful gap between answer accuracy and correct visual grounding in safety-critical multimodal systems.
VLESA: Vision-Language Embodied Safety Agent for Real-Time Human Activity Monitoring
Researchers introduce VLESA, a framework that monitors human activities from egocentric video and triggers real-time safety interventions when dangerous actions are predicted. The system addresses intent-dependent safety — where identical actions can be safe or dangerous depending on context — using a goal-conditioned safety Q-filter trained via GRPO and an intent-action prediction agent. On the ASIMOV-2.0 benchmark, VLESA achieves higher intervention accuracy than baselines, with the Q-filter improving action safety by over 41 percentage points through goal-conditioned constrained decoding.
GaP: Graph-as-Policy multi-agent harness for reliable robot automation via self-learning computation graphs
Researchers introduce Graph-as-Policy (GaP), a multi-agent coding harness that generates directed computation graphs combining perception, planning, and control nodes for robotic 'Variational Automation' tasks — those with high variation in object geometry and pose. GaP uses an internal simulation environment to rehearse and iteratively refine graph structures in parallel, improving success rates without relying solely on model-free policies. Evaluation across 8 new benchmarks (4 simulated, 4 real-world) shows significant outperformance over baselines. The work bridges agentic coding systems with interpretable robot programming and Task and Motion Planning.