Training Design for Text-to-Image Models: Lessons from Ablations
Photoroom shares practical lessons from ablation studies on training design choices for text-to-image diffusion models. The post covers decisions around data curation, model architecture, and training hyperparameters derived from systematic experimentation. This is part two of a series documenting Photoroom's internal research into building production-grade image generation systems.
Related guides (3)
Related events (8)
PRX Part 3 — Training a Text-to-Image Model in 24 Hours
Photoroom shares the third installment of their PRX series on Hugging Face, detailing how they trained a text-to-image model within a 24-hour window. The post covers the practical engineering and training infrastructure decisions that enabled rapid model development. This is part of an ongoing series documenting Photoroom's internal model development process.
Training Stable Diffusion with Dreambooth using Diffusers
This Hugging Face blog post describes how to fine-tune Stable Diffusion models using the DreamBooth technique via the Diffusers library. DreamBooth enables personalized text-to-image generation by training a model on a small set of reference images. The post covers the technical workflow for applying this fine-tuning approach within the Diffusers ecosystem.
Representation-Conditioned Diffusion Models for Controllable Image Generation
This paper explores conditioning diffusion models on representations from pre-trained self-supervised models as an alternative to text prompts or semantic maps, which require large annotated datasets. The self-conditioning mechanism improves unconditional image generation quality and provides a controllable representation space. The authors identify directions of variation in this space and demonstrate smoothness and disentanglement properties, suggesting potential for fine-grained generative control without heavy annotation overhead.
Instruction-tuning Stable Diffusion with InstructPix2Pix
This Hugging Face blog post describes a methodology for instruction-tuning Stable Diffusion using the InstructPix2Pix framework, enabling image editing via natural language instructions. The approach adapts techniques from language model instruction-tuning to the image generation domain. The post covers dataset construction, training procedures, and evaluation of the resulting models.
Meta Research Improves Image Generation via Staged Planning and Self-Revision Fine-Tuning
Researchers from Meta and collaborating universities propose a fine-tuning method that teaches image generators to compose images through discrete plan-sketch-inspect-refine cycles rather than generating all at once. Starting from BAGEL-7B, they construct ~62,000 training examples using GPT-4o and FLUX.1 Kontext to supervise each stage, achieving 83% on GenEval versus 77% for the base model and a competing method (PARM) that required 11x more training data and ~8x more inference steps. The approach improves spatial relationship accuracy, object attribute fidelity, and real-world knowledge grounding in generated images.
The Technology Behind BLOOM Training
This Hugging Face blog post details the infrastructure and training methodology used to train BLOOM, a 176-billion parameter open-access multilingual language model. It covers the use of Megatron-DeepSpeed for distributed training across hundreds of GPUs, including tensor parallelism, pipeline parallelism, and data parallelism strategies. The post also discusses hardware setup, memory optimization techniques, and lessons learned during the large-scale training run.
The Annotated Diffusion Model
A Hugging Face blog post providing a detailed, annotated walkthrough of diffusion models for image generation, likely covering the mathematical foundations and implementation details of denoising diffusion probabilistic models (DDPMs). The post serves as an educational deep-dive into the architecture and training process of diffusion-based generative models. Published in mid-2022, it coincides with the period of rapid growth in diffusion model adoption.
A Dive into Text-to-Video Models
A Hugging Face blog post providing an overview of text-to-video generation models as of mid-2023. The post surveys the landscape of approaches, architectures, and key models in the emerging text-to-video space. As a tier-2 commentary piece, it synthesizes existing work rather than presenting novel research.