faster-whisper

Reimplements OpenAI's Whisper ASR model using CTranslate2, a specialized inference engine for Transformer models that applies operator-level optimizations (graph compilation, memory pooling, quantization-aware kernels) to achieve 4x faster transcription than the original implementation while maintaining identical accuracy. The WhisperModel class wraps CTranslate2's compiled model format, enabling CPU and GPU inference with automatic device selection and fallback mechanisms.

BatchedInferencePipeline class implements a queue-based parallel processing architecture that groups multiple audio files into batches and processes them through the CTranslate2 inference engine simultaneously, achieving 3-5x additional speedup over sequential WhisperModel transcription. Uses dynamic batch sizing based on available GPU/CPU memory and implements work-stealing scheduling to balance load across processing threads.

Implements audio decoding using PyAV (Python bindings for FFmpeg libraries) bundled as a dependency, eliminating the need for separate FFmpeg installation. The decode_audio() utility supports 100+ audio formats (MP3, WAV, FLAC, M4A, OGG, OPUS, AIFF, etc.) and automatically resamples to 16kHz mono, handling format detection, channel mixing, and sample rate conversion in a single pass.

Provides model conversion utilities that transform OpenAI's PyTorch Whisper checkpoints into optimized CTranslate2 format, applying graph compilation, operator fusion, and quantization during conversion. The conversion process is one-time offline operation that generates hardware-optimized model files, enabling fast inference without requiring PyTorch at runtime.

Provides format_timestamp() utility and output formatting options that convert transcription results into standard subtitle formats (SRT, VTT) and JSON, with configurable timestamp precision and segment boundaries. The formatter handles edge cases like overlapping segments, missing timestamps, and language-specific formatting rules.

Integrates Silero VAD v6 model to detect speech segments and remove silence from audio before transcription, reducing processing time by ~50% by skipping non-speech regions. The VAD pipeline operates as a preprocessing stage that segments audio into speech/non-speech chunks, filters out silence, and passes only active speech regions to the Whisper encoder, reducing token count and inference cost.

Extracts word-level timestamps by analyzing cross-attention weights between the Whisper decoder and encoder outputs, mapping each decoded token to its corresponding audio time region. The mechanism leverages the Transformer's attention patterns to align subword tokens to audio frames, then aggregates token-level alignments into word-level boundaries without requiring external alignment models or post-processing.

Automatically detects the language of input audio by processing the first 30 seconds through Whisper's language identification head, which outputs probability scores across 99 supported languages. The detection runs as a lightweight preprocessing step before full transcription, enabling single-pass multilingual pipelines without requiring language hints or separate language detection models.

Supports 8-bit integer quantization (int8) and float16 precision modes during model loading, reducing model size by 35-50% and memory footprint proportionally while maintaining >99% accuracy. Quantization is applied at the CTranslate2 model conversion stage, not runtime, enabling hardware-accelerated quantized inference on CPUs and GPUs that support int8 operations.

Accepts a list of hotwords and optional prefix text that biases the Whisper decoder toward recognizing specific terms or continuing with expected text patterns. The biasing mechanism modifies token logits during beam search decoding, increasing probability of hotword tokens and prefix-consistent sequences, enabling domain-specific transcription without fine-tuning.

Implements beam search decoding with configurable beam width (default 5) and temperature-based fallback mechanism. If beam search fails to produce valid output (e.g., due to numerical instability), the decoder automatically falls back to temperature-sampled decoding with adjustable temperature parameter, ensuring robustness across diverse audio conditions without requiring user intervention.

Processes stereo audio by separating left and right channels and transcribing each independently, then merging results with channel labels to enable speaker diarization without external speaker separation models. The mechanism treats each channel as a separate audio stream, assigns speaker labels based on channel identity, and reconstructs timeline with speaker boundaries.

Provides download_model() utility that automatically fetches pre-converted CTranslate2 models from Hugging Face Hub, caches them locally with integrity verification, and manages model versioning. The caching mechanism uses content-addressable storage (hash-based paths) to prevent corruption and enable atomic updates, with configurable cache directory and automatic cleanup of unused models.