Diffusion Models
diffusion-models-5b88d83f·9 events·first seen 1mo agoAliases: Diffusion Models, diffusion model
Co-occurring entities
More like this (12)
Guides (1)
Recent events (9)
Representation-Conditioned Diffusion Models for Controllable Image Generation
This paper explores conditioning diffusion models on representations from pre-trained self-supervised models as an alternative to text prompts or semantic maps, which require large annotated datasets. The self-conditioning mechanism improves unconditional image generation quality and provides a controllable representation space. The authors identify directions of variation in this space and demonstrate smoothness and disentanglement properties, suggesting potential for fine-grained generative control without heavy annotation overhead.
Finite-Sample Lens for Understanding Diffusion Posterior Sampler Failures
This paper introduces a finite-sample theoretical framework for analyzing diffusion model posterior samplers used in imaging inverse problems. The authors show that popular likelihood approximations at intermediate timesteps systematically under- or over-estimate posterior spread, leading to failure modes including sensitivity to early stopping, incorrect weighting of posterior modes, and hallucination of prior or likelihood modes. Crucially, they demonstrate these failures can arise from a multimodal prior alone, without requiring nonlinear measurement models or multimodal posteriors. The framework is model-agnostic and can serve as a diagnostic tool for evaluating existing and future posterior samplers.
KLIP: Localized OOD Detection in Inverse Problems via KL-Divergence with Diffusion Priors
KLIP proposes an out-of-distribution detection metric for computational imaging that computes KL-divergence between a diffusion model prior and the posterior distribution. Unlike prior approaches, it requires no calibration data or knowledge of the shifted distribution, and can both flag whole images and localize OOD patches within images. The method is validated on medical imaging tasks such as detecting liver tumors in CT scans and generalizes across diffusion model architectures, datasets, and inverse problem types.
Simplifying, Stabilizing, and Scaling Continuous-Time Consistency Models
OpenAI has published research advancing continuous-time consistency models (sCMs), achieving sample quality comparable to leading diffusion models while requiring only two sampling steps. The work addresses prior instability and complexity issues in consistency model training. This represents a significant efficiency improvement for generative image synthesis, potentially enabling faster inference pipelines.
Consistency Models
OpenAI introduces Consistency Models, a new generative modeling framework designed to address the slow iterative sampling process inherent in diffusion models. The approach aims to enable faster single-step or few-step generation for image, audio, and video synthesis. The post appears to be a research announcement or blog summary of the underlying technique.
Review: Generative Models, Multimodal Learning, and Closed-Loop Workflows in Inverse Materials Design
This arxiv review surveys recent advances in generative modeling for inverse materials design, covering variational autoencoders, normalizing flows, autoregressive models, and diffusion models applied to crystalline solid discovery. It examines how multimodal learning fuses crystal structures, thermodynamic data, spectroscopy, microscopy, and scientific text into transferable chemical-space representations. The paper also reviews closed-loop design pipelines integrating conditional generation with Bayesian optimization, reinforcement learning, and active learning, and identifies recurring failure modes including surrogate exploitation, diversity collapse, and the stability-synthesizability gap.
Hierarchical Text-Conditional Image Generation with CLIP Latents (DALL-E 2 / unCLIP)
OpenAI published research on hierarchical text-conditional image generation using CLIP latents, the technique underlying DALL-E 2. The approach uses a prior network to map text embeddings to image embeddings, then a diffusion decoder to generate images from those embeddings. This represented a significant advance in text-to-image generation quality and semantic fidelity at the time of release.
Squeezing Capacity from MLLMs for Subject-driven Image Generation via Dual Layer Aggregation
This paper proposes conditioning diffusion models on Multimodal Large Language Models (MLLMs) that jointly encode text and reference images, augmented with VAE-based identity conditioning to address copy-paste artifacts and identity preservation failures in subject-driven image generation. A Dual Layer Aggregation (DLA) module aggregates multi-level MLLM features, and a multi-stage denoising strategy progressively balances semantic and fine-detail identity signals during inference. Experiments show improved human preference scores on subject-driven generation benchmarks compared to prior approaches that encode text and reference images separately.
Improved Techniques for Training Consistency Models
OpenAI presents improved training techniques for consistency models, a class of generative models capable of producing high-quality samples in a single step without adversarial training. The work advances a nascent alternative to diffusion-based generation that trades multi-step sampling for single-step inference. The post originates from OpenAI's research blog, indicating continued investment in efficient generative modeling.