How Diffusion Models Work - DeepLearning.AI

Provides step-by-step visual walkthroughs of how noise is progressively added to images during the forward diffusion process, using animated visualizations to show the mathematical transformation at each timestep. The course uses interactive Jupyter notebooks with rendered outputs to demonstrate how Gaussian noise accumulates according to a predefined noise schedule, making the abstract mathematical process concrete and observable.

Teaches the reverse diffusion process where a neural network learns to predict and remove noise iteratively, reconstructing images from pure Gaussian noise. The course explains the denoising network architecture, loss functions (mean squared error on noise prediction), and sampling strategies (DDPM, DDIM) through code walkthroughs and mathematical derivations, showing how the network learns to reverse the forward corruption process.

Demonstrates how to condition diffusion models on text embeddings to enable text-to-image generation, using techniques like cross-attention mechanisms to inject text information into the denoising network. The course explains how text encoders (CLIP, T5) produce embeddings that guide the reverse diffusion process, and covers classifier-free guidance to balance text adherence with image quality.

Teaches how to design and tune noise schedules (the variance curve controlling noise addition across timesteps) to optimize convergence speed and sample quality. The course covers linear, quadratic, and cosine schedules, explains their mathematical properties, and demonstrates empirically how schedule choice affects training dynamics and final image quality through comparative visualizations.

Walks through the complete training procedure for diffusion models, including data loading, noise injection at random timesteps, denoising network forward passes, loss computation (MSE on noise prediction), and backpropagation. The course provides end-to-end PyTorch code showing how to structure training loops, handle batch processing, and monitor training metrics specific to diffusion models.

Explains the U-Net architecture commonly used as the denoising network in diffusion models, covering encoder-decoder structure with skip connections, time embedding injection, and attention mechanisms. The course provides architectural diagrams and code implementations showing how timestep information is incorporated via sinusoidal embeddings and how spatial information is preserved through skip connections.

Teaches how to evaluate diffusion models using metrics like Fréchet Inception Distance (FID), Inception Score (IS), and LPIPS, explaining what each metric measures and how to interpret results. The course covers both distribution-level metrics (comparing generated and real image distributions) and perceptual metrics (measuring human-perceived quality), with code examples for computing these metrics on generated samples.

Teaches how to apply diffusion in latent space rather than pixel space by first encoding images using a variational autoencoder (VAE), performing diffusion on compressed latent representations, and decoding back to pixels. The course explains why latent diffusion is more efficient (smaller spatial dimensions, faster sampling), covers VAE architecture and training, and shows how to integrate pre-trained VAE encoders/decoders with diffusion models.

Covers techniques for adapting pre-trained diffusion models to new domains or styles without full retraining, including LoRA (Low-Rank Adaptation), textual inversion, and DreamBooth. The course explains how these methods reduce trainable parameters while maintaining model quality, provides code for implementing each technique, and discusses when to use each approach based on computational budget and desired adaptation level.