PHYRE

VAORA: Visual Action Outcome Reasoning Alignment for physical reasoning generalization in VLMs

Researchers introduce VAORA (Visual Action Outcome Reasoning Alignment), a reward design framework targeting two failure modes in vision-language models applied to interactive physical reasoning: hallucinated chain-of-thought reasoning and misalignment between reasoning and actions. VAORA uses two complementary rewards — a Visual Alignment Reward anchoring reasoning to visual context, and a Visual-Action Alignment Reward grounding reasoning in action outcomes — plus dense rewards from a pre-trained expert agent for training stability. Experiments on the PHYRE and Virtual Tool benchmarks show improved generalization to novel tasks and unseen environments.