Cleft

Converts spoken audio into text using on-device speech recognition models that never transmit audio data to external servers. The implementation leverages browser-native Web Speech API or local inference engines (likely ONNX Runtime or TensorFlow Lite) to perform acoustic-to-phoneme mapping and language modeling entirely within the user's device sandbox, eliminating cloud transmission overhead and ensuring audio payloads remain under user control.

Transforms raw transcribed text into semantically structured markdown by detecting natural speech patterns (pauses, emphasis, topic shifts) and converting them into markdown syntax (headers, lists, bold/italic, code blocks). The system likely uses NLP-based sentence segmentation, keyword extraction, and heuristic rules to infer document structure from spoken discourse patterns, outputting valid markdown that integrates directly with note-taking ecosystems.

Provides streaming transcription output as the user speaks, displaying partial results that update incrementally as new audio frames are processed. The implementation uses a streaming speech recognition pipeline (likely attention-based RNN or Conformer architecture) that processes audio chunks and emits intermediate hypotheses, allowing users to see text appear in real-time and make corrections before finalizing the note.

Exports transcribed and formatted notes to multiple target formats and platforms including markdown files, Obsidian vault integration, Notion API sync, and plain text. The system implements format-specific adapters that handle platform-specific metadata (Obsidian frontmatter, Notion block structure, Notion database properties) and provides direct API integrations or file-based exports depending on the target platform.

Indexes transcribed notes locally using a full-text search engine (likely SQLite FTS or similar embedded solution) to enable fast keyword-based retrieval without cloud indexing. The system builds an inverted index of note content, timestamps, and metadata, allowing users to search across all captured notes with sub-second latency entirely on their device.

Detects and labels different speakers in multi-speaker audio (meetings, interviews, group discussions) by analyzing voice characteristics and assigning speaker labels to transcribed segments. The implementation likely uses speaker embedding models (x-vectors or similar) to cluster voice patterns and assign consistent speaker IDs, then organizes note content by speaker for easier reference and attribution.

Maintains precise timestamp mappings between transcribed text segments and original audio, enabling users to click on any note text to jump to that point in the recording. The implementation stores segment-level timing metadata (start/end timestamps for each sentence or phrase) and provides playback controls synchronized with note content, allowing users to verify transcription accuracy by reviewing the original audio.

Enables voice note capture and transcription entirely offline, storing notes locally and automatically syncing to cloud platforms (Notion, Obsidian Sync, etc.) when network connectivity is restored. The implementation uses local-first architecture with conflict-free replicated data types (CRDTs) or similar patterns to handle offline edits and ensure consistency when syncing, allowing users to work without interruption regardless of connectivity.