Bark

Converts arbitrary text input to high-quality audio waveforms through a four-stage cascading pipeline: text→semantic tokens (80M transformer with causal attention), semantic→coarse audio structure (80M transformer), coarse→fine audio details (80M transformer with non-causal attention), and finally token→waveform via Facebook's EnCodec decoder. This architecture avoids phoneme dependencies and enables direct generative modeling of diverse audio types including speech, music, and sound effects.

Generates natural speech across 13 languages (English, Spanish, French, German, Italian, Portuguese, Polish, Turkish, Russian, Dutch, Czech, Chinese, Japanese) using a single unified transformer model trained on multilingual data. The text model tokenizes input with BERT and produces language-agnostic semantic tokens that the downstream coarse/fine models decode into language-appropriate audio, enabling zero-shot cross-lingual generation without language-specific model variants.

The fine transformer model uses non-causal (bidirectional) attention instead of causal attention, allowing it to attend to future audio tokens when predicting current tokens. This enables the model to refine audio details with full context of surrounding audio structure, improving coherence and naturalness compared to causal-only generation, while the coarse model uses causal attention to establish initial audio structure.

Enables speaker voice control by conditioning the generation pipeline on reference audio samples (history prompts). The system extracts acoustic characteristics from a reference audio file and uses these as conditioning context in the coarse and fine transformer models, allowing users to clone or adapt voices from 100+ preset voice samples or custom audio without explicit speaker embeddings or speaker ID training.

Extends generation beyond the default ~13-second context window by automatically splitting input text into chunks, generating audio for each chunk independently, and concatenating results with optional overlap handling to maintain prosodic continuity. The system manages chunk boundaries intelligently (at sentence/phrase breaks) and handles voice prompt carryover between chunks to maintain speaker consistency across long-form content.

Allows fine-grained control over audio output characteristics (laughter, singing, emphasis, emotional tone) by embedding special tokens directly in input text (e.g., '[laughter]', '[singing]'). These tokens are processed by the text model and propagated through the semantic token representation, influencing the coarse and fine models' output without requiring separate model variants or explicit style embeddings.

Provides three model size variants (full 80M-parameter, small 40M-parameter, minimal with CPU offloading) that automatically adapt to available hardware resources. The system can offload individual transformer layers to CPU during inference, enabling generation on devices with limited VRAM (2GB minimum) by trading computation speed for memory efficiency, with automatic layer scheduling to minimize data transfer overhead.

Represents audio as discrete tokens using Facebook's EnCodec neural codec (8 codebooks, 1,024 vocabulary per codebook), enabling the transformer models to operate on audio as a sequence of tokens rather than raw waveforms. The coarse model generates the first 2 codebooks (low-frequency structure), the fine model generates all 8 codebooks (full detail), and the EnCodec decoder reconstructs 24kHz audio from tokens with ~90dB SNR quality, enabling efficient transformer-based audio generation without spectrogram or waveform prediction.

Provides a high-level Python API (bark.generate_audio, bark.text_to_speech) that abstracts the four-stage cascade while exposing intermediate semantic token representations for debugging and analysis. Users can inspect semantic tokens generated by the text model, modify them before passing to coarse/fine models, or use tokens directly for downstream tasks, enabling both simple one-line generation and advanced token-level control.

Integrates with Hugging Face model hub to automatically download and cache pre-trained model weights on first use, eliminating manual weight management. The system uses huggingface_hub library to fetch text, coarse, fine, and codec models from suno-ai organization, with automatic caching in ~/.cache/huggingface/hub and fallback to local paths if available, enabling one-command setup without manual model downloads.

Tokenizes input text using BERT tokenizer and converts to BERT embeddings, which are then projected to semantic token space (10,000 vocabulary) by the text model. This approach provides language-agnostic text representation that works across 13 languages without language-specific tokenizers, enabling the same semantic token vocabulary to represent text in any supported language.