Twitter Spaces Downloader and Transcriber

Queries Twitter's API endpoints to check whether a given Spaces room is currently live or archived, and retrieves associated metadata including host information, participant count, and creation timestamp. Uses OAuth 2.0 authentication to access Twitter's v2 API and parses JSON responses to extract availability state and room identifiers for downstream processing.

Establishes persistent WebSocket or HTTP Live Streaming (HLS) connections to Twitter's Spaces audio infrastructure, buffers the incoming audio stream in real-time, and writes the complete broadcast to disk in standard audio formats (MP3, WAV, or M4A). Implements retry logic and connection recovery to handle network interruptions during long-running Spaces sessions.

Sends downloaded Spaces audio to a speech-to-text service (likely Whisper API or similar) with speaker diarization enabled, processes the returned transcript to identify and label individual speakers, and structures the output with timestamps and speaker attribution. Handles long-form audio by chunking into segments and managing context across chunks to maintain speaker consistency.

Converts transcription output into multiple standard formats (plain text, SRT subtitles, VTT captions, JSON with metadata, Markdown with speaker labels) using format-specific serialization logic. Preserves timestamps, speaker attribution, and confidence scores across all formats while applying format-appropriate styling and structure.

Analyzes transcript content and Spaces metadata to automatically extract and assign structured tags (topics, speakers, key themes) using keyword extraction or lightweight NLP. Enriches downloaded Spaces records with searchable metadata including duration, participant count, host, creation date, and AI-generated summaries or topic labels for catalog organization.

Manages sequential or parallel processing of multiple Spaces URLs through the complete pipeline (availability check → download → transcription → export → tagging) with progress tracking, error handling, and result aggregation. Implements job queuing and retry logic to handle failures gracefully and resume interrupted batch operations.

Indexes downloaded Spaces transcripts and metadata using full-text search and semantic similarity matching, enabling queries across transcript content, speaker names, topics, and timestamps. Supports both keyword search (regex or inverted index) and semantic search (embedding-based similarity) to find relevant Spaces by content or topic.

Processes Spaces transcripts through an LLM to generate structured summaries, extract key points, identify main topics, and produce actionable insights. Uses prompt engineering or few-shot examples to guide the LLM toward consistent, high-quality summaries with configurable detail levels (brief, standard, detailed).