MochiDiffusion

Executes Stable Diffusion image generation models directly on Apple Silicon's Neural Engine using Core ML framework, leveraging split_einsum model optimization to distribute computation across CPU, GPU, and Neural Engine. The pipeline chains multiple Core ML models (text encoder, UNet denoiser, VAE decoder) with custom scheduling logic to minimize memory footprint (~150MB) while maximizing throughput through hardware-specific compute unit selection.

Accepts an existing image as input and generates variations by injecting the reference image's latent representation into the diffusion process at a configurable noise level (strength parameter). The VAE encoder converts the input image to latent space, the UNet denoiser applies conditional diffusion starting from the noisy latent, and the VAE decoder reconstructs the final image. Strength parameter (0.0-1.0) controls how much the output diverges from the input: low values preserve composition, high values enable radical transformation.

Implements localization for UI strings, help text, and documentation in multiple languages (English, Chinese, Korean, etc.) using Xcode's localization system (.strings files and Localizable.strings). Language selection is automatic based on system locale but can be overridden in settings. All UI elements (buttons, labels, prompts) are localized; documentation is provided in multiple languages via README files.

Integrates Sparkle framework for automatic app updates, checking for new versions on app launch and periodically in background. Updates are downloaded silently and installed on next app restart with user notification. Update manifest (appcast.xml) is hosted on GitHub and specifies available versions, download URLs, and release notes. Users can manually check for updates or disable automatic checking in settings.

Enables users to import custom Core ML Stable Diffusion models from local directories without recompiling the app. The system scans a designated models directory (in app bundle or user Documents) for .mlmodel or .mlpackage files, automatically detects model type (split_einsum vs. original) and architecture (v1.5, v2.1, SDXL), and makes them available in the model selection UI. Model metadata (name, size, compute unit compatibility) is extracted from file attributes and model bundle info.

Integrates ControlNet models (separate Core ML networks) into the diffusion pipeline to provide structural guidance via edge maps, depth maps, pose skeletons, or other conditioning inputs. The ControlNet processes the conditioning image in parallel with the main UNet, producing cross-attention guidance that steers generation toward matching the structural constraints. Multiple ControlNet models can be loaded and weighted independently, enabling composition of multiple constraints (e.g., pose + depth).

Applies Real-ESRGAN neural network model (converted to Core ML) to generated or imported images to increase resolution by 2x or 4x while enhancing detail and reducing artifacts. The upscaler processes images in tiles to manage memory constraints, applies learned super-resolution kernels, and blends tile boundaries to avoid seams. Upscaling runs asynchronously in the job queue to avoid blocking UI.

Manages sequential or parallel image generation tasks in a queue system, tracking progress per job (step count, ETA, memory usage) and enabling cancellation mid-generation. Jobs are persisted to disk and survive app restart. The queue system decouples UI from long-running inference, allowing users to queue multiple generations and interact with the app while processing occurs. Progress updates stream to UI via SwiftUI state bindings.

Automatically embeds generation parameters (prompt, negative prompt, seed, model name, guidance scale, steps, ControlNet settings) into PNG/JPEG EXIF metadata when saving images. Metadata is human-readable and machine-parseable, enabling downstream tools to reproduce generations or extract parameters for analysis. Metadata is preserved when images are exported or shared.

Manages loading, caching, and selection of Core ML Stable Diffusion models with automatic compute unit assignment (CPU, GPU, Neural Engine). The system detects model type (split_einsum vs. original) and selects optimal compute unit based on model architecture and available hardware. Models are lazy-loaded on first use and cached in memory to avoid repeated disk I/O. Custom models can be imported from user-specified directories.

Implements multiple noise scheduling algorithms (DDPM, DDIM, Euler, Karras) that control the diffusion process across inference steps. The scheduler determines noise levels at each step, enabling trade-offs between quality and speed. Users can select scheduler and number of steps (typically 20-50); fewer steps reduce latency but may reduce quality. Scheduler is applied uniformly across all generation modes (text-to-image, image-to-image, ControlNet).

Implements native macOS user interface using SwiftUI framework with three main sections: gallery view (grid of generated images with metadata), sidebar controls (prompt input, model selection, generation parameters), and inspector panel (detailed image metadata and export options). UI is responsive to generation progress via SwiftUI state bindings, updating in real-time as jobs complete. Sidebar controls are context-aware, showing relevant options based on selected generation mode (text-to-image, image-to-image, ControlNet).

Manages persistent storage of generated images in app's Documents directory with SQLite or plist-based metadata index. Gallery view loads images lazily from disk, caching thumbnails in memory for fast scrolling. Images are organized by generation date and searchable by prompt text. Deletion removes both image file and metadata entry. Export functionality copies images to user-selected locations with metadata preservation.