metropolis-hastings-b15e973f·2 events·first seen Aliases: Metropolis-Hastings
BayesPO reframes prompt optimization as Bayesian posterior sampling over discrete prompt tokens, combining a task likelihood term with a language-model prior to form an energy-based objective. The framework uses a Metropolis-Hastings corrected Gibbs-with-Langevin proposal with parallel tempering to explore rugged LLM energy landscapes without updating model weights. Experiments on Qwen2.5 models show modest accuracy gains on instruction-induction tasks (60.04% to 63.23%), with identified limitations around overfitting and computational cost. The work positions principled probabilistic methods as an alternative to heuristic prompt search procedures.
This paper introduces Entropy-Cut Metropolis-Hastings (ECMH), an algorithm that samples from a 'power distribution' over base language model outputs to elicit strong reasoning without reinforcement learning posttraining. Rather than cutting reasoning traces at uniformly random positions, ECMH uses next-token entropy as a proxy to identify consequential decision points (e.g., choice of proof strategy), then resamples from those positions. The authors prove that mixing time scales with the number of decisions rather than tokens, and demonstrate consistent improvements over RL-trained models on MATH500, HumanEval, GPQA Diamond, and AIME26.