Qwen3-TTS-12Hz-0.6B-CustomVoice

Generates natural-sounding speech from text input across 12 languages (English, Chinese, Japanese, Korean, German, French, Russian, Portuguese, Spanish, Italian, and others) using a 600M parameter diffusion-based architecture. The model employs a two-stage pipeline: first converting text to acoustic features via a language-aware encoder, then synthesizing waveforms at 12Hz sampling rate using conditional diffusion. Custom voice cloning is achieved through speaker embedding injection, allowing users to condition generation on reference voice characteristics without full model fine-tuning.

Extracts speaker-specific embeddings from reference audio using a learned encoder that captures voice identity characteristics (timbre, pitch range, speaking patterns). These embeddings are injected into the diffusion conditioning mechanism during synthesis, allowing the model to reproduce voice characteristics without explicit prosody parameters. The embedding space is learned jointly with the TTS decoder, creating a continuous representation of speaker identity that generalizes across different phonetic contexts.

Processes input text through a language-aware encoder that handles language-specific tokenization, grapheme-to-phoneme conversion, and linguistic feature extraction for 12 languages. The encoder produces intermediate acoustic feature representations (mel-spectrograms or similar) that serve as conditioning input to the diffusion decoder. Language identification is implicit in the model architecture, allowing seamless handling of language-specific phonetic rules, tone marks (for tonal languages like Chinese), and diacritics without explicit language tags.

Generates audio waveforms using a conditional diffusion model that iteratively denoises random noise into coherent speech, conditioned on acoustic features and speaker embeddings. The diffusion process operates at 12Hz sampling rate, producing audio through a series of denoising steps (typically 50-100 steps) that progressively refine the waveform. Conditioning is applied through cross-attention mechanisms, allowing the model to incorporate both linguistic content (from text encoding) and speaker identity (from embeddings) throughout the generation process.

Supports efficient batch processing of multiple text inputs with automatic padding and masking to handle variable-length sequences. The implementation uses dynamic batching where sequences are grouped by length to minimize padding overhead, and attention masks ensure the model ignores padded positions. Inference can be optimized through step reduction (fewer diffusion steps for speed), mixed precision (float16 on compatible hardware), and optional gradient checkpointing to reduce memory usage during batch generation.

Provides optional post-processing capabilities to enhance generated audio quality, including normalization (peak normalization, loudness normalization to LUFS standard), noise reduction, and format conversion. The pipeline operates on generated waveforms before output, allowing users to standardize audio characteristics across multiple generations or adapt output to specific platform requirements (e.g., streaming services with loudness standards). Post-processing is modular and optional, allowing users to bypass it for raw model output.