Demucs music stem separator rewritten in Rust – runs in the browser

Executes the Demucs neural network model (vocals, drums, bass, other) directly in the browser using ONNX Runtime WebAssembly, eliminating server-side processing. The Rust codebase compiles to WebAssembly via wasm-bindgen, exposing a JavaScript API that loads pre-trained model weights and runs inference on client-side audio buffers without network latency or privacy concerns.

Handles chunked audio input processing by managing sliding windows of audio frames, buffering partial chunks, and coordinating inference timing to avoid gaps or overlaps in stem output. The Rust implementation uses ring buffers or deque structures to queue incoming audio data and emit inference-ready chunks at the model's required sample rate and frame size, with overlap-add reconstruction for seamless stem reconstruction.

Loads pre-trained Demucs model weights from ONNX format files and caches them in browser memory or IndexedDB to avoid re-downloading on subsequent uses. The implementation handles model initialization, weight tensor mapping to the inference graph, and optional persistent storage using browser APIs, with fallback to re-download if cache is unavailable.

Coordinates inference across multiple output stems (vocals, drums, bass, other) by running the Demucs model once per stem or using a multi-output model variant that produces all stems in a single forward pass. The Rust implementation manages tensor allocation, inference scheduling, and output collection to ensure all stems are computed and synchronized before returning results to the caller.

Converts input audio from various formats (MP3, WAV, WebM, OGG) to raw PCM buffers at the model's expected sample rate, handling codec decoding and sample rate conversion transparently. The implementation uses browser Web Audio API for decoding and Rust-based resampling (e.g., sinc interpolation or linear interpolation) to match the model's input requirements without requiring external libraries.

Encodes separated stems from raw PCM buffers into downloadable audio files (WAV, MP3, or other formats) with metadata (sample rate, bit depth, channel count). The implementation uses browser APIs or Rust-based encoders to convert Float32Array buffers to file formats, handling byte ordering, header generation, and optional compression.

Exposes callbacks or event emitters that report inference progress (e.g., percentage complete, current stem being processed) and allow users to cancel ongoing inference. The implementation divides inference into checkpoints, emits progress events after each checkpoint, and checks for cancellation signals before proceeding to the next step.

Catches inference errors (e.g., out-of-memory, invalid model, corrupted audio) and returns meaningful error messages to the caller, with optional fallback strategies (e.g., reduce audio quality, use smaller model variant). The implementation includes validation of input audio, model state checks, and error propagation through the JavaScript API.