🎙️ OpenSource Voice Dictation Agent (Wispr Flow clone)

Captures audio input via Fn key (hold-to-record) or double-tap (hands-free toggle) using a native C++ module (fn_key_monitor.node) that hooks into macOS keyboard events at the system level, bypassing Electron's renderer process limitations. The native module runs in the main process and communicates via IPC to trigger audio recording without application focus requirements, enabling dictation in any macOS application.

Implements a pluggable transcription architecture that routes audio to either local Whisper models (tiny/base/small via whisper-node-addon) for offline processing or cloud Deepgram API for high-speed transcription. The system abstracts transcription provider selection through a configuration layer, allowing users to toggle between privacy-first local processing and speed-optimized cloud processing without code changes. Audio is buffered in the renderer process and sent to the main process via IPC, which routes to the selected provider.

Implements a privacy-first architecture with zero telemetry — no analytics libraries, no tracking pixels, no data collection beyond what's necessary for core functionality. The app does not send usage data, crash reports, or user behavior analytics to any external service. All processing (transcription, LLM post-processing) can be done locally without cloud connectivity, and cloud processing (Deepgram, LLM APIs) only sends audio/text when explicitly configured by the user.

Extends Jarvis to iOS via a beta version distributed through Apple TestFlight, enabling voice dictation on iPhone and iPad. The iOS implementation (ios/README.md) uses native iOS APIs for audio capture and keyboard integration, with the same dual-path architecture (local Whisper or cloud Deepgram) as the macOS version. TestFlight allows beta testing with up to 10,000 external testers before App Store release.

Chains transcribed text through an LLM-based post-processing pipeline that removes filler words ('um', 'like', 'uh'), corrects grammar, adds punctuation, and enhances readability. The system supports dual-path LLM routing: local Ollama server (models: sam860/LFM2:1.2b, llama3, mistral) for offline processing or cloud LLMs (Gemini, Claude, OpenAI) for higher quality. Post-processing is triggered automatically after transcription completes, with results cached to avoid re-processing identical transcripts.

Implements a secure inter-process communication (IPC) bridge between Electron's main process (native module access, file I/O, API calls) and renderer process (UI, user interactions) using ipcMain and ipcRenderer with preload script isolation. The preload script (src/preload.ts) exposes a whitelist of safe IPC channels (e.g., 'start-recording', 'transcribe-audio', 'update-settings') to the renderer, preventing direct access to Node.js APIs and enforcing context isolation. Audio buffers and settings are marshaled through IPC as serialized JSON or binary data.

Persists user configuration (transcription provider, LLM choice, API keys, keyboard shortcuts) to disk using electron-store, a lightweight JSON-based key-value store that encrypts sensitive data (API keys) at rest. The onboarding interface (Onboarding Interface component) guides first-time users through provider selection (local vs cloud), API key configuration, and keyboard shortcut customization. Settings are loaded on app startup and cached in memory; changes trigger IPC updates to all processes and persist immediately to disk.

Captures audio from the system microphone using Web Audio API (in renderer process) or native audio APIs (via native modules in main process), with automatic gain control and noise suppression. Audio is buffered in memory as PCM samples at 16kHz sample rate, then sent to the transcription pipeline via IPC. The system handles microphone permission requests (macOS Privacy & Security) and gracefully degrades if microphone is unavailable or denied.

Builds and distributes native C++ modules (fn_key_monitor.node, whisper-node-addon) as separate binaries for Apple Silicon (ARM64) and Intel (x64) architectures using node-gyp and electron-builder. The build process compiles native code with platform-specific optimizations, signs binaries with Apple developer certificate, and packages both architectures into a universal macOS app. At runtime, the app loads the correct binary based on process.arch, ensuring optimal performance on each architecture.

Implements a dual-mode interaction model where users can either hold Fn to record (single-press mode) or double-tap Fn to toggle hands-free recording on/off (toggle mode). The native keyboard module (fn_key_monitor.node) detects double-tap timing (two presses within 300ms) and switches recording state accordingly. In hands-free mode, the app continues recording until the user double-taps again, enabling dictation without holding a key.

Provides a React-based settings interface (Settings Interface component) where users can select transcription provider (local Whisper vs Deepgram), LLM provider (local Ollama vs cloud APIs), and configure API keys securely. The settings UI is rendered in a separate Electron window and communicates with the main process via IPC to read/write settings to electron-store. Sensitive fields (API keys) are masked in the UI and encrypted at rest in the settings file.

Automates code signing and Apple notarization of the macOS app bundle using electron-builder and custom build scripts. The build process signs the app with a valid Apple developer certificate, submits it to Apple's notarization service, and waits for approval before packaging the DMG installer. Notarized apps display no security warnings when users download and launch them, improving trust and reducing support burden.