browser-based meeting platform joining with platform-specific automation
Enables AI agents to join Google Meet, Zoom, and Microsoft Teams meetings through Playwright-based browser automation with platform-specific controllers that handle each platform's unique UI patterns, authentication flows, and meeting state management. The BrowserMeetingProvider abstracts platform differences while delegating to GoogleMeetController, ZoomController, and TeamsController for platform-specific interactions, managing virtual display (Xvfb) and audio device routing.
Unique: Uses modular platform-specific controllers (GoogleMeetController, ZoomController, TeamsController) that encapsulate UI interaction logic per platform, allowing independent updates without affecting other platforms. Manages virtual display and audio routing at the provider level, abstracting infrastructure complexity from agent code.
vs alternatives: More maintainable than monolithic browser automation because platform logic is isolated in controllers; more flexible than API-only solutions because it works with any meeting platform that has a web interface
real-time audio capture and voice activity detection pipeline
Captures audio from meeting participants in real-time through PulseAudio integration and applies Voice Activity Detection (VAD) to filter silence and background noise before sending to transcription. The DefaultTranscriptionController orchestrates the VAD → STT pipeline, using pluggable VAD service providers (local or cloud-based) to reduce transcription costs by only processing segments with actual speech.
Unique: Implements pluggable VAD service architecture allowing runtime selection between local (privacy-preserving) and cloud-based VAD providers, with configurable sensitivity thresholds. Integrates directly with PulseAudio for low-level audio device control rather than relying on higher-level audio libraries.
vs alternatives: More cost-effective than transcribing all audio because VAD pre-filters silence; more privacy-preserving than cloud-only solutions because local VAD options are available; more flexible than fixed VAD implementations because providers are swappable
client sdk with joinlyclient api for agent development
Provides high-level Python SDK (joinly-client package) with JoinlyClient class that abstracts MCP communication and session management, enabling developers to build meeting agents without understanding MCP protocol details. SDK handles connection lifecycle, tool calling, and transcript streaming, providing a simple async API for agent code.
Unique: Abstracts MCP protocol complexity through a high-level JoinlyClient API, enabling developers to build agents with simple async methods (join_meeting, send_message, get_transcript) without MCP knowledge. Integrates ConversationalToolAgent for LLM-based agent logic.
vs alternatives: More developer-friendly than raw MCP because abstractions hide protocol details; more integrated than generic MCP clients because it understands meeting-specific operations natively
shared type system and protocol definitions for cross-package consistency
Defines shared data types (Transcript, AudioFormat, AudioChunk) and service provider protocols in joinly-common package, ensuring consistent interfaces across server and client packages. Protocols define expected behavior for VAD, STT, and TTS providers, enabling type-safe provider implementations and reducing integration errors.
Unique: Uses Python protocols to define service provider interfaces (VAD, STT, TTS) without requiring inheritance, enabling flexible provider implementations while maintaining type safety. Shared types (Transcript, AudioFormat) ensure consistent data representation across server and client.
vs alternatives: More flexible than inheritance-based interfaces because protocols support structural typing; more maintainable than duplicated type definitions because shared types are defined once in joinly-common
speech-to-text transcription with pluggable provider support
Converts filtered audio segments to text using configurable STT service providers (e.g., OpenAI Whisper, Google Cloud Speech, local models). The DefaultTranscriptionController receives VAD-filtered audio chunks and routes them to the selected STT provider, returning Transcript objects with text, confidence scores, and timing metadata for agent consumption.
Unique: Abstracts STT provider selection through a pluggable service architecture, allowing runtime provider switching via configuration without code changes. Maintains Transcript data type across all providers, ensuring consistent downstream agent integration regardless of STT backend.
vs alternatives: More flexible than single-provider solutions because agents aren't locked into one STT service; more maintainable than custom provider wrappers because the framework handles provider lifecycle and error handling
text-to-speech synthesis with real-time audio output
Converts agent text responses to speech and outputs audio to the meeting in real-time using configurable TTS service providers (e.g., Resemble, Google Cloud TTS, local TTS engines). The DefaultSpeechController manages the TTS → audio output pipeline, handling audio format conversion, buffering, and PulseAudio device routing to ensure agent speech is heard by meeting participants.
Unique: Implements pluggable TTS provider architecture (e.g., Resemble.ai integration in joinly/services/tts/resemble.py) with audio format conversion and PulseAudio sink management, allowing provider swapping without agent code changes. Handles real-time audio buffering and synchronization with meeting audio stream.
vs alternatives: More flexible than single-provider TTS because voice quality and cost can be optimized per deployment; more integrated than generic TTS libraries because it handles meeting-specific audio routing and synchronization
mcp-based meeting tool exposure for llm agents
Exposes meeting capabilities (join, transcribe, speak, get participants, etc.) as standardized Model Context Protocol (MCP) tools that LLM agents can call. The FastMCP server interface wraps meeting operations as callable tools with JSON schemas, enabling any MCP-compatible LLM client to interact with meetings through a standard protocol without needing to understand Joinly's internal APIs.
Unique: Implements FastMCP server that wraps Joinly's meeting operations as standardized MCP tools, enabling any MCP-compatible LLM to control meetings without custom integrations. Uses Server-Sent Events for real-time updates (transcripts, participant changes) alongside request-response tool calls.
vs alternatives: More interoperable than proprietary APIs because MCP is a standard protocol; more maintainable than custom LLM integrations because tool schemas are defined once and work across all MCP clients
session management and dependency injection for meeting orchestration
Manages meeting session lifecycle (creation, state tracking, resource cleanup) through the MeetingSession orchestrator class, using dependency injection to wire together platform providers, audio controllers, and service implementations. Sessions maintain state across multiple operations, handle concurrent audio processing, and ensure proper resource cleanup on meeting termination.
Unique: Uses dependency injection pattern to wire together platform providers, audio controllers, and service implementations, allowing flexible composition without tight coupling. MeetingSession acts as central orchestrator coordinating browser automation, audio processing, and transcription pipelines.
vs alternatives: More maintainable than monolithic session handling because concerns are separated; more testable because dependencies can be mocked; more flexible because service implementations can be swapped without changing session code
+4 more capabilities