brainrot.js

Generates full debate-format videos between multiple public figures by orchestrating a pipeline that accepts user-provided debate prompts, routes them through an LLM to generate dialogue scripts with speaker attribution, converts each speaker's lines to speech using pre-trained RVC (Retrieval-based Voice Conversion) models fine-tuned on celebrity voice samples, synchronizes audio tracks, and renders final video output using Remotion with character animations. The system maintains separate voice models per public figure (stored in training_audio/ directory) and uses tRPC API endpoints to manage the generation workflow across distributed backend services.

Accepts a user-provided topic or debate prompt and routes it through an LLM (ChatGPT via API) to generate multi-turn dialogue scripts with explicit speaker labels and turn-taking structure. The system parses LLM output to extract speaker names, dialogue lines, and optional stage directions, then validates speaker names against the pre-trained voice model registry before passing to the TTS pipeline. This ensures generated scripts only reference available voice models and maintains consistent speaker identity throughout the video.

Implements a specialized video mode (monologue) that generates single-speaker narration from a topic prompt, with the LLM generating a coherent speech from one character's perspective. The system renders monologue videos with full-screen character focus and optional background visuals, enabling character-driven storytelling without multi-speaker dialogue. Monologue mode is optimized for faster rendering (shorter videos, single audio track) and lower LLM costs (single speaker generation).

Implements asynchronous video rendering via a job queue stored in the pendingVideos database table, with CI/CD pipeline (.github/workflows/deploy-ec2.yml) that deploys rendering workers to AWS EC2 instances. When a user requests video generation, the system enqueues a job in pendingVideos, and distributed EC2 workers poll the queue, claim jobs, execute the Remotion rendering pipeline, upload completed videos to S3, and update the videos table. This architecture decouples user requests from rendering latency, enabling horizontal scaling without blocking the API.

Packages RVC voice conversion service in a Docker container (rvc/Dockerfile) with Python dependencies (rvc/requirements.txt), enabling isolated, reproducible deployment of the voice conversion backend. The container runs RVC inference with GPU support (NVIDIA CUDA), accepts audio input via HTTP API, performs voice conversion, and returns converted audio. Docker containerization decouples RVC from the main Node.js backend, allowing independent scaling and updates.

Stores generated MP4 video files in AWS S3 buckets with signed URLs for secure, time-limited access. The system uploads completed videos from EC2 rendering workers to S3, stores S3 URLs in the videos database table, and generates signed URLs (valid for 1 hour) for user downloads. S3 can be configured with CloudFront CDN for geographic distribution and faster delivery to users worldwide.

Converts generic text-to-speech audio (generated via Speechify API) into celebrity-specific voices by running inference on pre-trained RVC (Retrieval-based Voice Conversion) models. Each public figure has a dedicated RVC model trained on their voice samples (stored in training_audio/ directory), and the system loads the appropriate model based on speaker selection, applies voice conversion to the TTS audio, and outputs character-specific speech. The RVC backend runs in a Docker container (rvc/Dockerfile) with Python dependencies (rvc/requirements.txt) and is orchestrated via tRPC API calls from the main backend.

Orchestrates video rendering using Remotion (React-based video framework) to compose character animations, background visuals, and synchronized audio tracks into a final MP4 file. The system defines React components for each video mode (debate, podcast, monologue, rap) that accept dialogue scripts and audio files as props, renders frames at specified FPS, and outputs video with audio sync. Rendering is triggered via tRPC API endpoint (src/app/api/create/route.ts) and can be distributed across multiple EC2 instances via a job queue (pendingVideos table) to handle concurrent requests.

Integrates Speechify API (generate/speechifyAudioGenerator.ts) to convert dialogue text into generic speech audio before voice conversion. The system sends dialogue lines to Speechify with specified voice parameters (gender, speed, pitch), receives MP3 audio files, and passes them to the RVC voice conversion pipeline. This two-stage approach (generic TTS → RVC voice conversion) enables character-specific voices without requiring custom voice models for every possible speaker.

Implements tRPC (TypeScript RPC framework) API layer (src/server/api/routers/users.ts, src/trpc/shared.ts) that exposes video generation endpoints with type-safe request/response contracts. The API routes user requests through a state machine: validate user credits, queue video generation job in pendingVideos table, trigger backend services (LLM dialogue generation, TTS, RVC, Remotion rendering), poll job status, and return completed video metadata. tRPC provides end-to-end type safety between Next.js frontend and backend, eliminating runtime type mismatches.

Implements authentication via Next.js auth middleware (src/app/layout.tsx, src/app/providers.tsx) with session management and a credit system that tracks user video generation quota. Users authenticate via email/password or OAuth, and each video generation request deducts credits from the brainrotusers table. The system enforces credit checks before queuing videos, preventing over-quota usage. Stripe integration enables credit purchases and subscription management, with webhook handlers updating user credit balances on successful payment.

Stores completed videos in a videos database table with metadata (video_id, user_id, title, duration, speaker_list, s3_url, created_at) and provides API endpoints to list, retrieve, and delete user videos. The system tracks video ownership via user_id foreign key, enabling per-user video libraries accessible via src/app/yourvideos.tsx component. Videos are stored as MP4 files in AWS S3 with signed URLs for secure access, and metadata is queryable for search/filtering.

Implements a specialized video mode (rap) that generates rap lyrics via LLM, synthesizes rap vocals with beat-matched timing, and renders video synchronized to background music. The system accepts a topic and music track, generates rap lyrics with rhyme scheme and meter, converts lyrics to speech with timing metadata, and overlays rap audio onto background music track in Remotion. The rapAudio table tracks rap-specific audio files and beat synchronization metadata, enabling precise timing between vocals and instrumental.

Implements a specialized video mode (podcast) that generates longer-form dialogue between multiple speakers with discussion-style turn-taking, topic transitions, and conversational flow. The LLM prompt is optimized for podcast dialogue (longer turns, follow-up questions, tangential discussions) rather than debate-style quick exchanges. Remotion renders podcast videos with speaker panels or interview-style layouts, and the system supports longer video durations (10-30 minutes) compared to short-form debate videos (1-3 minutes).