stable-diffusion-webui

Generates images from natural language text prompts by encoding prompts through CLIP text encoder, then conditioning the Stable Diffusion UNet denoising process across multiple sampling steps. The pipeline processes prompts into embeddings, applies guidance scaling (classifier-free guidance), and iteratively denoises latent representations using configurable samplers (DDIM, Euler, DPM++, etc.) before decoding to pixel space via VAE decoder. Supports negative prompts, prompt weighting syntax, and dynamic prompt scheduling across generation steps.

Transforms existing images by encoding them into latent space via VAE encoder, then conditioning the diffusion process to preserve structural features while applying style/content modifications. The pipeline injects the encoded image at a configurable denoising step (controlled by 'denoising strength' parameter: 0-1), allowing users to control how much of the original image is preserved vs regenerated. Supports inpainting masks to selectively regenerate regions, and outpainting to extend image boundaries with coherent content generation.

Generates multiple images in a single request with deterministic reproducibility via seed control. The system accepts batch size parameter, generates images sequentially or in parallel, and uses seed values to ensure identical outputs for identical inputs. Supports seed increment (seed, seed+1, seed+2, etc.) for variations on a theme, or fixed seed for exact reproduction. Batch results are returned as list of images with metadata (seed, parameters) for each image.

Upscales images using multiple passes of img2img generation with decreasing denoising strength, progressively refining details while maintaining composition. The system supports both built-in upscalers (RealESRGAN, BSRGAN, SwinIR) and diffusion-based upscaling via repeated img2img passes. Each pass applies a small amount of denoising to add detail without drastically altering the image. Supports arbitrary upscaling factors (2x, 4x, 8x) and custom upscaler selection.

Provides browser-based graphical interface built with Gradio framework, enabling non-technical users to generate images without command-line interaction. The UI includes real-time progress bars showing generation progress, intermediate step previews (optional), and live parameter adjustment. Components are organized into tabs (txt2img, img2img, inpainting, etc.) with collapsible sections for advanced parameters. The UI automatically serializes user inputs to generation parameters and displays results with metadata (seed, parameters, generation time).

Automatically detects Stable Diffusion model architecture (1.5, 2.0, 2.1, XL, custom) from checkpoint metadata or weights, and routes to appropriate processing pipeline. The system inspects model dimensions (UNet channels, text encoder size, VAE architecture) to determine compatibility and required processing steps. Supports both standard architectures and custom fine-tunes with automatic fallback to compatible pipeline. Enables seamless switching between different model versions without manual configuration.

Manages loading, caching, and switching between multiple Stable Diffusion model checkpoints (1.5, 2.1, XL, custom fine-tunes) with automatic VRAM optimization. The system discovers checkpoints from configured directories, maintains a model cache to avoid redundant disk I/O, and implements memory-efficient loading via half-precision (fp16) or 8-bit quantization. Supports checkpoint metadata parsing (model type, VAE variant, training dataset) and automatic architecture detection to route to appropriate processing pipeline.

Loads and composes Low-Rank Adaptation (LoRA) modules and textual inversion embeddings into the base model without modifying checkpoint weights. LoRA adapters inject learnable low-rank matrices into UNet and text encoder layers, enabling style/subject control via weight merging. Textual inversions replace single tokens with learned embedding vectors, allowing concept injection via prompt syntax (e.g., '<my-style>'). The system supports multiple simultaneous LoRA adapters with per-adapter strength scaling, and automatic discovery of embeddings from configured directories.

Provides abstraction over 15+ diffusion samplers (DDIM, Euler, DPM++, Heun, LMS, etc.) and noise schedulers (linear, cosine, sqrt, Karras) with configurable step counts and guidance scaling. Each sampler implements different numerical integration schemes for the diffusion ODE, trading off speed vs quality. The system allows per-step noise schedule adjustment, dynamic guidance scaling, and sampler-specific parameters (e.g., DPM++ uses second-order corrections). Samplers are swappable without model retraining, enabling rapid experimentation.

Exposes all generation capabilities via RESTful HTTP endpoints with JSON request/response serialization. The API layer wraps the image generation pipeline, accepting prompts, parameters, and images as multipart form data or JSON, and returning generated images as base64-encoded PNG or raw binary. Supports both synchronous blocking requests and asynchronous job submission with polling. Implements request validation, error handling with descriptive HTTP status codes, and optional authentication via API key.

Provides extensibility via Python scripts that hook into the generation pipeline at predefined callback points (before/after processing, before/after sampling, etc.). Extensions can modify prompts, parameters, or intermediate outputs without modifying core code. The system discovers scripts from the scripts/ directory, exposes them in the UI as tabs or buttons, and passes processing context (images, prompts, parameters) to script functions. Supports both UI scripts (with Gradio components) and backend scripts (pure Python).

Generates grid of images by varying one, two, or three parameters (X, Y, Z axes) across specified ranges, enabling systematic exploration of parameter space. The system creates a matrix of generation requests with different parameter combinations, executes them sequentially or in batches, and arranges results in a grid with axis labels. Supports varying any generation parameter (prompt, guidance_scale, sampler, steps, seed, etc.) and produces a single composite image or individual tiles for analysis.

Trains custom textual inversion embeddings by optimizing a learnable token embedding vector to match a concept using a small dataset of images (5-20 images typical). The training loop iterates over images, encodes them via VAE, adds noise, and optimizes the embedding to predict the noise using the diffusion model. The system includes dataset preparation utilities (image resizing, augmentation) and hyperparameter controls (learning rate, iterations, regularization). Trained embeddings are saved as .pt files and can be loaded into any Stable Diffusion model.

Allows loading alternative VAE models to replace the default VAE decoder/encoder used for latent space compression. Different VAEs produce different aesthetic qualities (some preserve detail, others smooth artifacts). The system supports VAE swapping without reloading the full checkpoint, caching VAE models in memory, and applying VAE-specific optimizations (tiling for large images, sliced attention for memory efficiency). Supports both standard VAEs and specialized variants (VAE-FT-MSE for detail, VAE-FT-EMA for smoothness).