5arXiv cs.CL (Computation and Language)·
Theoretical unification of discrete diffusion model objectives via Oracle Distance theorem
A new arXiv preprint provides a rigorous theoretical framework for understanding what discrete diffusion models learn, proving the 'Oracle Distance' theorem: the negative ELBO exactly equals data entropy plus the path KL from the oracle reverse process to the learned one. The work shows that denoiser, score ratio, and bridge plug-in parameterizations are the same object in different coordinates, with closed-form conversions among them. It unifies several existing discrete diffusion losses (MDM, UDM, SEDD, GIDD) as special cases and identifies practical consequences, such as why denoiser parameterization causes the uniform ELBO to diverge at initialization. All identities are verified numerically on an exactly solvable model.
Related guides (1)
Related events (8)
The Annotated Diffusion Model
A Hugging Face blog post providing a detailed, annotated walkthrough of diffusion models for image generation, likely covering the mathematical foundations and implementation details of denoising diffusion probabilistic models (DDPMs). The post serves as an educational deep-dive into the architecture and training process of diffusion-based generative models. Published in mid-2022, it coincides with the period of rapid growth in diffusion model adoption.
KLIP: Localized OOD Detection in Inverse Problems via KL-Divergence with Diffusion Priors
KLIP proposes an out-of-distribution detection metric for computational imaging that computes KL-divergence between a diffusion model prior and the posterior distribution. Unlike prior approaches, it requires no calibration data or knowledge of the shifted distribution, and can both flag whole images and localize OOD patches within images. The method is validated on medical imaging tasks such as detecting liver tumors in CT scans and generalizes across diffusion model architectures, datasets, and inverse problem types.
Kolmogorov Regression lifts diffusion policies to Cameron-Martin space for robust long-horizon control
Researchers introduce a backward Kolmogorov equation framework that reformulates diffusion policy training as a deterministic boundary-value PDE problem in Cameron-Martin space, replacing stochastic score matching. The approach uses a precision-weighted Cameron-Martin loss and a Kolmogorov residual as an inference-time failure detector, yielding convergence guarantees tied to kernel effective rank rather than action dimension. Validation on the PushT manipulation benchmark shows 17% improvement in episode reward and 67.6% reduction in inter-step drift; a 6-station manufacturing scheduling task shows 28.4% lower RMSE than LSTM baselines and 96% reduction in deadlock events via Hamilton-Jacobi reachability certification.
Perturbation Theory for Spherical Hellinger-Kantorovich Flows with Differential Privacy Guarantees
This paper develops a perturbation theory for Spherical Hellinger-Kantorovich (SHK) gradient flows, which couple transport and reaction dynamics and coincide with birth-death Langevin dynamics. The authors derive dimension-free bounds on log-likelihood ratios and Rényi/KL divergences when two potentials differ, quantifying how perturbations propagate over time. These results are applied to differential privacy: the likelihood-ratio control yields explicit Pure-DP guarantees for SHK-based samplers implementing the exponential mechanism, while KL bounds provide Approximate-DP certificates. A utility bound is also derived that separates intrinsic exponential-mechanism suboptimality from finite-time sampling error.
Diffusion-Proof: First framework applying diffusion LLMs to formal theorem proving
Researchers introduce Diffusion-Proof, the first framework to train and apply diffusion language models (dLLMs) for formal theorem proving, addressing limitations of autoregressive models in long-range coherence. The framework includes dLLM-Prover-7B for whole-proof generation and dLLM-Corrector-7B for local proof correction via bidirectional infilling. Diffusion-Proof achieves absolute improvements of 1.61% on ProofNet-Test and 6.14% on MiniF2F-Test over an AR baseline, and solves one IMO problem that DeepSeek-Prover-V2-7B could not. The result suggests dLLMs may have structural advantages over AR models for tasks requiring long-range logical coherence.
Minimax framework establishes fundamental statistical limits for transport map estimation in generative models
A new arXiv preprint formalizes the task of estimating valid transport maps (used in diffusion models, normalizing flows, and flow matching) within a minimax statistical framework. The key result is that under standard stability assumptions from optimal transport theory, estimating any valid transport map is as statistically hard as estimating the optimal transport map itself. The authors also show that when those stability assumptions fail, alternative transport maps can be learned substantially more accurately than the OT map, clarifying when targeting sub-optimal maps yields real statistical advantages. The work provides theoretical sample complexity lower bounds applicable to a broad class of modern generative modeling methods.
RePlaid: Continuous Diffusion Language Models Scale Competitively with Discrete Diffusion
This paper revisits continuous diffusion language models (DLMs) by introducing RePlaid, an updated version of Plaid that aligns its architecture with modern discrete DLMs. RePlaid establishes the first scaling law for continuous DLMs competitive with discrete approaches, achieving a compute gap of only 20× versus autoregressive models and a state-of-the-art perplexity bound of 22.1 on OpenWebText among continuous DLMs. The authors provide theoretical analysis showing that likelihood-based training naturally yields linear cross-entropy over time and creates structured embedding geometries, explaining the performance gains.
Finite-Sample Lens for Understanding Diffusion Posterior Sampler Failures
This paper introduces a finite-sample theoretical framework for analyzing diffusion model posterior samplers used in imaging inverse problems. The authors show that popular likelihood approximations at intermediate timesteps systematically under- or over-estimate posterior spread, leading to failure modes including sensitivity to early stopping, incorrect weighting of posterior modes, and hallucination of prior or likelihood modes. Crucially, they demonstrate these failures can arise from a multimodal prior alone, without requiring nonlinear measurement models or multimodal posteriors. The framework is model-agnostic and can serve as a diagnostic tool for evaluating existing and future posterior samplers.