ComfyUI CLI

ComfyUI represents image generation pipelines as directed acyclic graphs where nodes represent atomic operations (model loading, sampling, conditioning, etc.). The execution engine traverses this graph, executing only nodes whose inputs have changed since the last run, leveraging a smart caching system that tracks node outputs and invalidates downstream dependencies. This architecture enables iterative refinement of complex multi-stage pipelines without re-executing unchanged operations, dramatically reducing inference latency for workflow modifications.

ComfyUI provides a plugin system where custom nodes are registered via Python classes implementing a standard interface (INPUT_TYPES, RETURN_TYPES, execute methods). The extension system dynamically discovers and loads custom nodes from designated directories, allowing third-party developers to add new operations without modifying core code. Each node declares its input/output types using a type system (comfy_types/node_typing.py) that enables automatic validation, UI generation, and workflow serialization.

ComfyUI supports video generation through specialized nodes for frame-by-frame generation, temporal consistency enforcement, and frame interpolation. The system can generate videos by iteratively sampling frames with temporal conditioning that maintains consistency across frames, or by generating keyframes and interpolating between them. Supports video models like Flux Video and WAN (World Animation Network) with specialized sampling strategies for temporal coherence.

ComfyUI implements a blueprint system that allows users to encapsulate complex subgraphs as reusable components with defined inputs and outputs. Blueprints are essentially workflows-within-workflows that can be instantiated multiple times with different parameters, enabling modular workflow design and code reuse. The system supports nested blueprints, parameter passing, and automatic input/output exposure.

ComfyUI provides a comprehensive CLI interface (cli_args.py, main.py) that allows headless execution of workflows without the web UI. The CLI supports specifying model paths, VRAM optimization flags, execution parameters, and workflow input overrides. The system can run in server mode (with API) or direct execution mode, enabling integration into automated pipelines and batch processing systems.

ComfyUI implements dynamic quantization strategies that automatically convert model weights to lower precision (FP16, INT8, NF4) based on available VRAM and user preferences. The system supports mixed-precision execution where different layers run at different precisions, and can dynamically switch precision during execution based on memory pressure. Quantization is applied transparently without requiring model retraining.

ComfyUI implements intelligent model loading (model_management.py, model_detection.py) that automatically detects model architecture, quantization format, and optimal device placement (CUDA/ROCm/CPU) based on available VRAM and model size. The system supports multiple quantization schemes (fp32, fp16, int8, NF4) and can dynamically offload models between VRAM and system RAM or disk based on memory pressure, using a priority-based eviction strategy to keep frequently-used models resident.

ComfyUI implements a sophisticated conditioning system that combines multiple control signals (text embeddings, image conditioning, ControlNet spatial guidance, T2I-Adapter features) into a unified conditioning tensor that guides the diffusion process. The system supports weighted combination of multiple conditioning inputs, negative conditioning for guidance inversion, and advanced guidance methods (CFG, DPM++ guidance) that modulate the denoising trajectory based on combined conditioning signals.

ComfyUI abstracts diffusion sampling into composable components: schedulers (noise schedules like linear, cosine, karras) that define the denoising trajectory, samplers (DPM++, Euler, Heun, etc.) that implement specific integration methods, and custom sampler nodes that allow users to define arbitrary sampling loops. The system decouples noise scheduling from sampling algorithm, enabling users to combine any scheduler with any sampler and implement novel sampling strategies without modifying core code.

ComfyUI implements a model patching system that applies Low-Rank Adaptation (LoRA) weights to base models by injecting learned low-rank updates into specific layers (typically attention and MLP layers). The system supports multiple LoRA files simultaneously with per-LoRA strength scaling, automatic layer matching across different model architectures, and efficient in-place weight modification that avoids duplicating the base model in memory.

ComfyUI abstracts VAE (Variational Autoencoder) operations through a latent format system (latent_formats.py) that handles encoding images to latent space and decoding latents back to images. The system supports multiple VAE variants (SD1.5, SDXL, Flux VAEs with different scaling factors) and latent formats (standard, tiled for memory efficiency, scaled variants), automatically selecting the appropriate VAE based on the base model and handling format conversions transparently.

ComfyUI exposes a REST/WebSocket API (server.py) that allows remote clients to submit workflows, monitor execution progress in real-time, and retrieve results. The API uses JSON workflow serialization, WebSocket connections for live progress updates (execution status, intermediate images, memory usage), and supports batch job submission with queuing. The system maintains execution history and allows clients to cancel running jobs or modify queued jobs.

ComfyUI implements text encoding through a pluggable encoder system that supports CLIP (OpenAI's vision-language model) as the primary text encoder, with support for alternative encoders like T5 (used in Flux) and other transformer-based models. The system handles tokenization, embedding generation, and prompt weighting (e.g., (prompt:1.5) syntax) that allows users to emphasize specific words or phrases in the generated output.

ComfyUI provides a comprehensive set of image processing nodes that handle resizing, cropping, blending, masking, and batch operations on images and masks. The system supports efficient batch processing where multiple images are processed simultaneously, and provides mask-aware operations that preserve transparency and alpha channels. Operations include interpolation methods (bilinear, nearest, lanczos), color space conversions, and compositing with alpha blending.