technique
ELBO variance minimization
techniqueactive
elbo-variance-minimization-6e63bb86·1 events·first seen 28d agoAliases: ELBO variance minimization
Co-occurring entities
More like this (12)
posterior predictive variance minimizationInvariant Risk MinimizationBayesian OptimizationMLE-benchVariance ReductionMaximum Likelihood Estimationolmo-evalPrivate Stochastic Convex Optimizationreward-induced maximum likelihoodDivergence Regularized Policy Optimizationamortized variational inferenceVariational Information Bottleneck
Recent events (1)
RePlaid: Continuous Diffusion Language Models Scale Competitively with Discrete Diffusion
This paper revisits continuous diffusion language models (DLMs) by introducing RePlaid, an updated version of Plaid that aligns its architecture with modern discrete DLMs. RePlaid establishes the first scaling law for continuous DLMs competitive with discrete approaches, achieving a compute gap of only 20× versus autoregressive models and a state-of-the-art perplexity bound of 22.1 on OpenWebText among continuous DLMs. The authors provide theoretical analysis showing that likelihood-based training naturally yields linear cross-entropy over time and creates structured embedding geometries, explaining the performance gains.