whisper.cpp

Executes OpenAI's Whisper model entirely on CPU using quantized weights and optimized matrix operations, eliminating GPU dependency. Implements GGML (Georgi Gerganov's Machine Learning) tensor library with hand-optimized kernels for x86, ARM, and WASM architectures, achieving real-time or near-real-time transcription on consumer hardware through aggressive quantization (Q4, Q5, Q8 formats) and memory-mapped model loading.

Automatically detects spoken language from audio and transcribes in 99+ languages using Whisper's multilingual encoder-decoder architecture. The model learns language-agnostic acoustic representations in the encoder, then uses language tokens to condition the decoder, enabling zero-shot transfer to languages unseen during fine-tuning. Language detection happens via a 50-token language classifier embedded in the model.

Provides a comprehensive CLI tool for running Whisper inference with extensive configuration options, including model selection, input/output format specification, language hints, timestamp generation, and performance tuning. The CLI supports both single-file and batch processing modes, with configuration via command-line flags, environment variables, or config files. Includes progress reporting, error handling, and output formatting options.

Provides pre-trained Whisper models optimized for specific languages (English-only variants) with reduced model size and improved accuracy for that language. The English-only models remove the multilingual encoder and language token logic, reducing parameters by ~30% and improving English transcription accuracy by 2-3%. Available in multiple sizes (tiny, base, small, medium, large) with corresponding quantization levels.

Generates transcription output with precise word-level and segment-level timestamps by leveraging Whisper's decoder attention patterns and cross-attention to the encoder. The implementation extracts timing information from the model's internal attention weights during inference, mapping each decoded token back to its corresponding audio frame, then aggregates frames into word and segment boundaries using heuristic post-processing.

Processes continuous audio streams in fixed-size chunks (e.g., 30-second windows) with overlap to maintain context, enabling near-real-time transcription without waiting for complete audio. The implementation buffers incoming audio samples, triggers inference when a chunk is ready, and uses overlapping windows to preserve word boundaries and context across chunk boundaries. Partial results are emitted as chunks complete, with final results refined as more context arrives.

Converts full-precision Whisper models (PyTorch, ONNX) to quantized GGML format with multiple precision levels (Q4_0, Q4_1, Q5_0, Q5_1, Q8_0) using a custom quantization pipeline. The process includes weight quantization (reducing 32-bit floats to 4-8 bits), layer-wise statistics collection for optimal quantization ranges, and format serialization into memory-mapped binary files. Supports both symmetric and asymmetric quantization strategies with per-channel or per-tensor granularity.

Parallelizes Whisper inference across multiple CPU cores using thread-pool-based work distribution at the tensor operation level. The implementation partitions matrix multiplications and element-wise operations across threads, with each thread processing a slice of the computation. Uses lock-free work queues and NUMA-aware thread pinning for optimal cache locality on multi-socket systems. Supports configurable thread count and automatic detection of available cores.

Compiles whisper.cpp to WebAssembly using Emscripten, exposing a JavaScript API for running Whisper inference directly in web browsers. The implementation includes audio capture from microphone/file input, model loading from IndexedDB or network, and streaming results back to JavaScript. Uses Web Workers to prevent blocking the main thread, with SharedArrayBuffer for efficient audio buffer passing between JS and WASM.

Processes multiple audio files sequentially or in parallel batches with automatic queue management, progress tracking, and error handling. The implementation maintains a work queue of audio files, distributes them to worker threads or processes, aggregates results, and provides callbacks for progress updates and error recovery. Supports priority queuing, retry logic for failed transcriptions, and output batching to reduce I/O overhead.

Applies audio signal processing to normalize and prepare raw audio for Whisper inference, including resampling to 16kHz, mono conversion, volume normalization (peak or RMS-based), silence trimming, and optional noise reduction. The implementation uses efficient DSP algorithms (polyphase resampling, FFT-based filtering) to minimize preprocessing latency. Supports multiple input sample rates (8kHz-48kHz) and channel configurations (mono, stereo, multi-channel).

Implements memory-mapped model loading and intelligent caching to avoid reloading models between inference runs. The implementation uses mmap to load quantized models without decompression, caches loaded models in memory with LRU eviction, and supports lazy loading of model layers on first use. Enables efficient multi-model scenarios (e.g., different languages or sizes) without excessive memory usage.