masked-diffusion-models-2ab3a5c3·3 events·first seen Aliases: Masked Diffusion Models, Masked Diffusion Model
Researchers introduce MDM-VGB, a discrete diffusion sampler for Masked Diffusion Models that augments token unmasking with reward-guided remasking inspired by the Jerrum-Sinclair backtracking Markov chain. The method extends backtracking from a fixed prefix tree to a masked-state graph, enabling tokens to be unmasked and remasked at arbitrary positions to favor higher-reward partial configurations. The authors prove quadratic complexity and robustness to process-verifier noise, contrasting with exponential complexity of best-of-N heuristics, and validate on constraint-satisfaction benchmarks including Sudoku and QM9.
Researchers introduce FMLM+, a framework combining Flow Map Language Models with masking-style noise schedules to enable joint sequence generation with per-token global consistency scoring. The key contribution is Posterior Refinement, an inference-time self-correction strategy that matches discrete baseline performance with 32x fewer neural function evaluations (NFEs). The approach improves the speed-quality tradeoff over both Masked Diffusion Models and standard FLMMs across multiple benchmarks, addressing longstanding factorization error problems in non-autoregressive generation.
LoopMDM introduces selective looping of early-middle transformer layers in masked diffusion language models, achieving a depth-scaling effect without adding parameters. The approach matches same-size MDM performance with up to 3.3× fewer training FLOPs and outperforms deeper non-looped MDMs on reasoning benchmarks, including up to 8.5 points improvement on GSM8K. Inference-time compute scaling is enabled by varying loop counts, with adaptive loop scheduling providing additional efficiency gains. Attention analysis suggests looping works by promoting interactions among masked token positions.