Auto-Encoding Variational Bayes (VAE)

Enables efficient inference over continuous latent variables in directed probabilistic models by reformulating the variational lower bound (ELBO) into a differentiable objective that decouples the sampling operation from gradient computation. Uses the reparameterization trick to transform intractable posterior expectations into deterministic transformations of continuous random variables, allowing end-to-end optimization via standard stochastic gradient descent without requiring specialized variational inference algorithms.

Learns compressed latent representations of data by training an encoder network to map high-dimensional inputs to a lower-dimensional latent space, then training a decoder to reconstruct the original input from latent codes. The reconstruction objective (likelihood term in ELBO) forces the latent space to capture task-relevant structure, while the KL divergence regularizer prevents the encoder from ignoring the latent variables. This produces interpretable, continuous embeddings suitable for downstream tasks like clustering, visualization, or generation.

Applies stochastic gradient descent with mini-batches to optimize the variational lower bound (ELBO) for latent variable models, avoiding the need for expensive full-dataset E-step computations required by classical EM or mean-field variational inference. The reparameterization trick enables low-variance gradient estimates from mini-batches, allowing convergence with modest batch sizes. This approach scales to datasets with millions of examples by processing small subsets at a time, making it practical for modern large-scale applications.

Generates new data samples by sampling latent codes from a simple prior distribution (e.g., standard Gaussian) and passing them through the learned decoder network. The prior is chosen to be tractable and easy to sample from, while the decoder learns to map latent codes to realistic data samples. This enables principled generation of new examples from the learned data distribution, with the ability to interpolate between samples by moving smoothly through latent space.

Learns an inference network (encoder) that approximates the intractable posterior distribution p(z|x) with a tractable variational approximation q(z|x). The encoder outputs parameters of a simple distribution (e.g., Gaussian with diagonal covariance) that approximates the true posterior. This enables efficient inference of latent variables given observations, allowing practitioners to discover latent factors of variation in data without requiring expensive inference algorithms or sampling methods.

Evaluates model quality using the evidence lower bound (ELBO), which decomposes into reconstruction loss (how well the model explains data) and KL divergence (how well the posterior matches the prior). The ELBO provides a principled, differentiable objective that balances model fit and regularization, enabling comparison of different architectures, hyperparameters, and model variants. Unlike ad-hoc metrics, the ELBO has a clear probabilistic interpretation as a lower bound on data likelihood.