simulation-aided experimental policy (SEP)

Mind the Sim-to-Real Gap & Think Like a Scientist: Fisher-SEP for Simulation-Aided Experimental Policy

This paper studies when and how a planner should supplement a pre-trained simulator with real-world experiments in sequential decision problems. The authors decompose simulator value error into a calibration-deployment shift (identifiable via randomization) and an irreducible parametric residual, and show that purely passive learning cannot close the reachability component of the value gap. They propose Fisher-SEP, a simulation-aided experimental policy that minimizes posterior predictive variance of a target policy's value, with case studies in supply chain and HIV mobile-testing domains demonstrating regimes where designed exploration is necessary.