text-to-image texture generation with stable diffusion
Generates 2D textures from natural language prompts by integrating Hugging Face Diffusers pipeline directly into Blender's UI layer. The DreamTexture operator collects prompt parameters (text, negative prompt, seed, guidance scale, steps) from a DreamPrompt property group, launches a background generator process to avoid blocking Blender's UI, and pipes the diffusers output directly into Blender's image editor. Supports multi-platform GPU acceleration (CUDA, DirectML, MPS, ROCm) with automatic device selection and fallback to CPU.
Unique: Runs Stable Diffusion as a background subprocess within Blender's Python environment rather than via external API or separate application, eliminating network latency and cloud dependencies while maintaining Blender UI responsiveness through async task management and progress callbacks.
vs alternatives: Faster iteration than cloud-based tools (no API round-trip) and more integrated than standalone generators, with native Blender material assignment and history tracking via DreamPrompt property groups.
image-to-image texture refinement with strength control
Modifies existing textures or images by passing them through the Stable Diffusion img2img pipeline with configurable denoising strength. The operator accepts an input image from Blender's image editor, applies the diffusers img2img pipeline with user-defined strength (0-1 scale controlling how much the original is preserved), and outputs a refined texture. Supports negative prompts and all generation parameters (seed, steps, guidance) to enable fine-grained control over stylization vs. preservation.
Unique: Integrates img2img as a first-class operation within Blender's texture workflow, allowing artists to toggle between text-to-image and img2img modes via the same DreamPrompt configuration without context switching to external tools.
vs alternatives: More seamless than Photoshop plugins or standalone img2img tools because the input/output remain in Blender's native image editor and material system, enabling direct application to 3D models.
animation re-styling with custom render pass integration
Applies AI-generated textures to animation frames by integrating with Blender's render engine and custom render passes. The operator renders animation frames with a custom pass (e.g., diffuse color, normal map), passes each frame through the img2img pipeline with a consistent prompt and seed offset, and outputs a re-styled animation. Maintains temporal coherence by using frame-based seed offsets and optical flow guidance to minimize flickering between frames.
Unique: Integrates custom render passes directly into the animation pipeline, allowing artists to apply img2img to specific render layers (diffuse, normal, etc.) rather than final composited frames, enabling more precise control over which aspects of the animation are re-styled.
vs alternatives: More flexible than external video processing tools because it operates on Blender's native render passes, enabling layer-specific styling and maintaining integration with Blender's material and lighting system.
node-based texture generation with custom render engine
Enables procedural texture generation workflows by implementing a custom Blender render engine that integrates Stable Diffusion into the Shader Editor node system. Artists can create node graphs with DreamTexture nodes (text-to-image, img2img, upscale, etc.), connect them to material outputs, and render to generate textures procedurally. Supports node inputs for prompts, parameters, and conditioning images, enabling complex multi-stage generation pipelines.
Unique: Implements a custom Blender render engine that treats Stable Diffusion operations as renderable nodes, enabling procedural texture generation within Blender's native node system rather than as separate operators.
vs alternatives: More powerful than operator-based workflows because node graphs enable complex multi-stage pipelines and reusable templates, whereas operators are single-stage and require manual chaining.
model management with automatic downloading and caching
Manages Stable Diffusion model weights by automatically downloading, caching, and versioning models from Hugging Face. The operator queries available models, downloads selected models on first use, caches them locally to avoid re-downloading, and manages disk space by allowing users to delete unused models. Supports multiple model variants (base, inpainting, upscaling, ControlNet) with independent caching.
Unique: Implements automatic model downloading and caching via Hugging Face's diffusers library, eliminating manual model setup and enabling seamless model switching without re-downloading.
vs alternatives: More convenient than manual model management because models are downloaded on-demand and cached automatically, whereas manual setup requires users to download and place models in specific directories.
performance optimization with memory-efficient inference
Optimizes generation speed and memory usage through multiple techniques: mixed-precision inference (float16 on GPU), attention slicing to reduce peak memory, model quantization, and VAE tiling for high-resolution outputs. The operator in `optimizations.py` applies these techniques based on available VRAM, enabling generation on lower-end GPUs (4GB) that would otherwise fail. Supports progressive optimization levels (aggressive, balanced, quality) for user control.
Unique: Implements automatic optimization selection based on detected VRAM, applying mixed-precision, attention slicing, and VAE tiling transparently without user configuration, whereas most tools require manual optimization tuning.
vs alternatives: More accessible than manual optimization because it automatically selects optimization levels based on hardware, enabling users with limited VRAM to generate textures without technical knowledge of inference optimization.
depth-aware texture generation with depth-to-image
Generates textures that respect 3D geometry by using depth maps as conditioning input to the Stable Diffusion pipeline. The operator extracts or accepts a depth map (from Blender's depth render pass or external source), passes it alongside the text prompt to the diffusers DepthToImagePipeline, and produces a texture that aligns with the geometric structure. Enables AI-generated textures to follow surface contours and relief patterns.
Unique: Bridges Blender's native depth rendering with Stable Diffusion's depth conditioning by automatically extracting depth from render passes, eliminating manual depth map export/import steps and enabling single-click depth-aware generation.
vs alternatives: More integrated than external depth-to-image tools because depth maps are generated directly from Blender's scene, ensuring perfect alignment with 3D geometry without manual alignment or coordinate transformation.
inpainting and outpainting with mask-based editing
Enables selective texture modification by accepting a mask image that defines which regions to regenerate. The operator loads a mask (white = regenerate, black = preserve) alongside the base image and prompt, passes both to the diffusers inpainting pipeline, and outputs a texture with only masked regions modified. Supports outpainting (extending textures beyond original boundaries) by expanding the canvas and masking the new regions.
Unique: Integrates mask-based inpainting directly into Blender's image editor workflow, allowing artists to paint masks using Blender's native brush tools and immediately apply inpainting without external mask creation tools.
vs alternatives: More efficient than manual retouching or external inpainting tools because masks are created and applied within Blender's unified interface, reducing tool-switching and enabling rapid iteration on texture edits.
+6 more capabilities