AssemblyAI API

Converts pre-recorded audio to text using AssemblyAI's Universal-3 Pro model, trained on 12.5+ million hours of audio data. Supports context-aware prompting via plain-language instructions and keyterms (up to 1000 words/phrases, max 6 words per phrase) to control transcription behavior. Provides word-level timestamps, speaker role identification, code-switching support, and verbatim mode. Processes audio asynchronously via REST API with per-hour-of-audio billing ($0.21/hr for Universal-3 Pro, $0.15/hr for legacy Universal-2 supporting 99 languages).

Provides real-time transcription of live audio streams using Universal-3 Pro model via WebSocket-based streaming API. Supports speaker role identification (by name or role, not generic diarization labels) and is built on AssemblyAI's proprietary Voice AI stack optimized for production voice agents. Processes audio with sub-second latency for interactive applications like live call transcription, voice agent interactions, and real-time meeting captions. Billed at $4.50/hr of audio processed.

Enables customization of transcription output by providing domain-specific terminology, custom spellings, or keyterms that should be recognized and preserved in the transcript. Supports up to 1000 words/phrases with a maximum of 6 words per phrase. Implemented as a prompting feature that influences the transcription model's output without requiring model fine-tuning. Billed at $0.05/hr of audio processed for Universal-3 Pro (included in base price) and $0.05/hr for Universal-2. Enables accurate transcription of specialized vocabulary, proper nouns, product names, and domain-specific terminology.

Applies large language models (LLMs) directly to audio data via AssemblyAI's LeMUR (Language Model on Embedded Representations) framework, enabling AI-powered tasks like summarization, question-answering, entity extraction, and custom analysis without requiring separate transcript processing. Processes audio through the transcription pipeline and applies LLM reasoning directly on the transcript representation. Specific LLM models supported, pricing, and integration details not documented in available material. Enables end-to-end audio intelligence workflows without chaining multiple services.

Transcription mode that preserves filler words, false starts, and non-standard speech patterns exactly as spoken, without normalization or cleanup. Implemented as a transcription parameter that disables automatic filler word removal and speech normalization, returning a verbatim record of the audio content. Useful for linguistic analysis, legal documentation, or accessibility applications requiring exact speech representation. Included in base transcription cost (no additional billing).

Handles audio containing multiple languages mixed within a single conversation (code-switching), accurately transcribing each language segment and optionally identifying language boundaries. Implemented as a native feature of Universal-3 Pro that detects language switches and transcribes each segment in the appropriate language. Enables accurate transcription of multilingual conversations without requiring separate language-specific models or manual language selection. Specific language pair support and language detection accuracy not documented in available material.

Provides precise timing information for each word in the transcript (start and end timestamps) along with per-word confidence scores indicating transcription accuracy. Implemented as a native feature of the transcription output that returns word-level metadata for synchronization with audio/video playback, interactive transcript building, or quality analysis. Enables downstream applications like interactive transcripts, video captions, and transcript-based search with playback seeking.

Returns precise word-level timing information for each word in the transcript, enabling applications to synchronize text with audio playback, highlight words as they're spoken, or extract segments by time range. Timestamps are returned in milliseconds with start and end times per word.

Segments transcript by speaker and assigns speaker labels to each segment, enabling identification of who said what in multi-speaker audio. Implemented as an add-on feature to both Universal-3 Pro and Universal-2 models, processing audio asynchronously and returning speaker-labeled segments in the transcript JSON response. Billed at $0.02/hr of audio processed (in addition to base transcription cost). Does not require pre-configuration of speaker count or identities.

Automatically detects and redacts personally identifiable information (PII) from transcripts, including person names, company names, email addresses, dates, and locations. Implemented as a speech understanding add-on feature that processes the transcript output and masks or removes sensitive entities. Returns redacted transcript with optional entity metadata for compliance and privacy workflows. Specific masking strategy (replacement tokens, hashing, removal) not documented in available material.

Analyzes transcript content for policy violations, inappropriate language, or flagged content categories. Implemented as a speech understanding add-on feature that processes the transcript and returns moderation scores or flags for content categories. Specific moderation categories, confidence thresholds, and flagging logic not documented in available material. Enables content filtering workflows for platforms with community guidelines or compliance requirements.

Generates abstractive summaries of transcribed audio content, extracting key points, main topics, and action items from the full transcript. Implemented as a speech understanding add-on feature that processes the transcript and returns a structured summary. Specific summarization algorithm (extractive vs abstractive), summary length control, and key point extraction logic not documented in available material. Enables rapid content review and knowledge extraction from long-form audio.

Analyzes the emotional tone and sentiment of transcript segments, detecting positive, negative, or neutral sentiment and optionally emotion categories (confidence, frustration, satisfaction, etc.). Implemented as a speech understanding add-on feature that processes the transcript and returns sentiment scores per segment or speaker. Specific emotion categories, scoring methodology, and segment granularity not documented in available material. Enables sentiment-driven insights from customer interactions, user research, and team communications.

Specialized transcription mode optimized for medical and healthcare conversations, with enhanced recognition of medical terminology, drug names, anatomical terms, and healthcare-specific vocabulary. Implemented as an add-on feature to both Universal-3 Pro and Universal-2 models, processing audio asynchronously and returning transcripts with improved accuracy for medical content. Billed at $0.15/hr of audio processed (in addition to base transcription cost). Enables compliance with healthcare documentation standards (HIPAA, medical record requirements).

Automatically detects and extracts named entities from transcripts, including person names, company names, email addresses, dates, and locations. Implemented as a native feature of the transcription output that identifies entity boundaries and types without requiring separate NLP processing. Returns entity metadata with positions in the transcript for downstream processing, indexing, or knowledge base construction. Enables rapid information extraction from unstructured audio content.

Identifies filler words and non-speech sounds (um, uh, ah, like, you know, etc.) in transcripts and optionally removes or flags them. Implemented as a native feature of the transcription output that detects filler words at the word level and returns them with position metadata. Enables transcript cleanup for professional documentation, presentation materials, or speech analysis. Specific filler word list and detection methodology not documented in available material.