fitted-occupancy-ratio-evaluation-without-bellman-completeness-be3c2486·1 events·first seen Aliases: Fitted Occupancy-Ratio Evaluation without Bellman Completeness
A new arXiv preprint introduces Fitted Occupancy-Ratio Evaluation (FORE), a fixed-point method for estimating discounted occupancy ratios in offline reinforcement learning and off-policy evaluation. FORE uses an adjoint Bellman recursion and projects onto a log-ratio class in KL divergence, requiring only realizability of the occupancy ratio rather than Bellman completeness or projected-operator stability. The authors establish finite-sample regret bounds and show convergence in KL up to approximation and statistical error, supporting value estimation via reward reweighting, occupancy-weighted fitted Q-evaluation, and doubly robust estimation. The result identifies occupancy-ratio realizability as a sufficient condition for offline policy evaluation, relaxing a standard assumption in the field.