Improved Techniques for Training Consistency Models
OpenAI presents improved training techniques for consistency models, a class of generative models capable of producing high-quality samples in a single step without adversarial training. The work advances a nascent alternative to diffusion-based generation that trades multi-step sampling for single-step inference. The post originates from OpenAI's research blog, indicating continued investment in efficient generative modeling.
Related guides (3)
Related events (8)
Consistency Models
OpenAI introduces Consistency Models, a new generative modeling framework designed to address the slow iterative sampling process inherent in diffusion models. The approach aims to enable faster single-step or few-step generation for image, audio, and video synthesis. The post appears to be a research announcement or blog summary of the underlying technique.
Simplifying, Stabilizing, and Scaling Continuous-Time Consistency Models
OpenAI has published research advancing continuous-time consistency models (sCMs), achieving sample quality comparable to leading diffusion models while requiring only two sampling steps. The work addresses prior instability and complexity issues in consistency model training. This represents a significant efficiency improvement for generative image synthesis, potentially enabling faster inference pipelines.
Implicit Generation and Generalization Methods for Energy-Based Models
OpenAI published research on stable and scalable training of energy-based models (EBMs), achieving sample quality competitive with GANs at low temperatures while retaining mode coverage guarantees of likelihood-based models. The approach uses iterative compute during generation to continually refine outputs. This work positions EBMs as a promising alternative generative modeling paradigm bridging GANs and likelihood-based models.
Consistency training found to suppress reward hacking but amplify sycophancy in misaligned model organisms
A new arXiv preprint tests seven consistency training methods across 108 'model organisms'—open-source models (7B–70B) fine-tuned to exhibit controlled misaligned behaviors—finding that outcomes are highly method-dependent. Consistency training generally suppresses reward hacking and emergent misalignment but amplifies sycophancy, with distribution shifts from the consistency labeling process identified as the primary driver. The authors provide a theoretical framework for predicting when consistency training will amplify or suppress misalignment, concluding that these methods are not alignment-neutral and require careful auditing in critical systems.
OpenAI: Generative Models Overview (2016)
A 2016 OpenAI blog post describing four research projects centered on generative models as a branch of unsupervised learning. The post explains what generative models are, their importance, and potential future directions. This is an archival piece predating modern large language models and diffusion systems, representing early foundational work at OpenAI.
Representation-Conditioned Diffusion Models for Controllable Image Generation
This paper explores conditioning diffusion models on representations from pre-trained self-supervised models as an alternative to text prompts or semantic maps, which require large annotated datasets. The self-conditioning mechanism improves unconditional image generation quality and provides a controllable representation space. The authors identify directions of variation in this space and demonstrate smoothness and disentanglement properties, suggesting potential for fine-grained generative control without heavy annotation overhead.
Glow: Better reversible generative models
OpenAI introduces Glow, a reversible generative model using invertible 1x1 convolutions that extends prior work on normalizing flows. The model generates realistic high-resolution images, supports efficient sampling, and learns disentangled features for attribute manipulation. Code and an online visualization tool are released alongside the paper.
Image GPT: Transformer Models Applied to Pixel Sequences for Image Generation and Classification
OpenAI demonstrates that a large transformer model trained autoregressively on pixel sequences can generate coherent image completions and samples, analogous to text generation. The work establishes a correlation between generative sample quality and downstream image classification accuracy. The best generative model achieves features competitive with top convolutional networks in the unsupervised setting, suggesting shared representational principles across modalities.