Graph Sparse Sampling

Graph Sparse Sampling (GSS) breaks exponential horizon dependence in continuous MDP planning

Researchers introduce Graph Sparse Sampling (GSS), an online planning algorithm for continuous Markov Decision Processes that shares sampled futures across candidate decisions rather than branching separately per action, eliminating the tree structure of methods like MCTS. The approach exposes large GPU-friendly batches and uses heuristics to focus computation, with finite-sample performance guarantees showing polynomial (rather than exponential) dependence on planning horizon under suitable conditions. Empirical results on continuous-control tasks show GSS substantially outperforms tree-based planners at long horizons. The work formalizes when shared-future graph planning can avoid the curse of the horizon that afflicts sparse sampling trees.