onyx

Onyx implements a pluggable connector framework that abstracts 20+ data sources (Slack, Google Drive, Confluence, GitHub, etc.) into a unified document ingestion pipeline. Each connector implements a standardized lifecycle (credential validation, document fetching, chunking, metadata extraction) and feeds into a Celery-based background task queue that coordinates with Vespa for full-text and semantic indexing. The system maintains connector state, handles incremental syncs, and manages credential encryption via a centralized credential store.

Onyx implements a RAG pipeline that retrieves relevant documents from Vespa using hybrid search (BM25 + semantic similarity), ranks results using LLM-based relevance scoring, and injects retrieved context into the LLM prompt with explicit citation metadata. The system tracks which documents contributed to each response, enables users to click through to source documents, and supports configurable retrieval strategies (dense-only, sparse-only, or hybrid). Retrieved chunks maintain document ID, source connector, and chunk position for precise citation.

Onyx provides a Next.js-based chat UI that streams LLM responses in real-time using Server-Sent Events (SSE), displaying tokens as they arrive. The frontend maintains local state for conversations, messages, and UI elements (input field, citation popups, research progress) using React hooks and TypeScript. The UI supports markdown rendering, code syntax highlighting, citation links, and responsive design. Real-time updates are coordinated via WebSocket or polling, and the frontend implements optimistic updates for better perceived latency.

Onyx implements a Model Context Protocol (MCP) server that exposes Onyx capabilities (search, retrieval, assistant management) to external LLM clients. External applications can call Onyx tools via MCP, enabling workflows where an external LLM orchestrates Onyx operations. The MCP server is implemented as a separate service that communicates with the main Onyx API, and supports standard MCP tool schemas for function calling. This enables integration with other AI systems and agents that support MCP.

Onyx provides an embeddable chat widget that can be deployed on third-party websites via a simple script tag. The widget communicates with the Onyx backend via CORS-enabled API calls and maintains conversation state in the browser. The widget is customizable (colors, position, initial message) via configuration parameters, and supports authentication via JWT tokens or API keys. The widget is built with vanilla JavaScript (no framework dependencies) to minimize bundle size and compatibility issues.

Onyx provides a desktop application (built with Electron or similar) that can run locally or connect to a remote Onyx instance. The desktop app maintains local conversation history and can work offline with cached documents. It supports keyboard shortcuts, system tray integration, and native file dialogs for document upload. The app is built with the same frontend code as the web UI, enabling code reuse and consistent UX across platforms.

Onyx provides a command-line interface (onyx-cli) for programmatic access to Onyx capabilities: searching documents, creating conversations, managing assistants, and uploading documents. The CLI is built with Python and uses the Onyx API, enabling automation workflows and integration with shell scripts. The CLI supports output formatting (JSON, CSV, table) for easy parsing, and authentication via API keys or environment variables.

Onyx provides a Chrome extension that enables searching Onyx documents and chatting with Onyx directly from the browser. The extension adds a sidebar to the browser that communicates with the Onyx backend, allowing users to search without leaving their current page. The extension supports authentication via OAuth or API keys, and maintains conversation state across browser sessions. The extension can be configured to search specific assistants or document collections.

Onyx implements a multi-turn research workflow where the LLM can iteratively refine queries, retrieve additional documents, and synthesize findings across multiple retrieval rounds. The system maintains conversation context, tracks which documents have been retrieved, and prevents redundant searches. Each research iteration generates a new query, retrieves fresh results, and updates the synthesis. This is coordinated via the chat message processing flow with state maintained in PostgreSQL conversation records.

Onyx abstracts LLM provider differences (OpenAI, Anthropic, Ollama, Azure, etc.) through a unified factory pattern that normalizes API calls, token counting, and error handling. The system implements a model selection hierarchy where assistants can specify preferred models, fallback models, and provider-specific configurations. LiteLLM is used as the underlying abstraction layer with custom monkey patches for Onyx-specific behavior (cost tracking, token limits, provider-specific prompt formatting). Each LLM provider has configurable access controls and quota limits enforced at the API server level.

Onyx allows creation of custom assistants with configurable system prompts, model selection, retrieval behavior, and tool bindings. Each assistant is stored as a database record with versioning, and can be assigned to users or organizations. Assistants can be configured to use specific LLM providers, retrieval strategies (dense/sparse/hybrid), and can bind to external tools via a schema-based function registry. Prompt templates support variable injection ({context}, {user_query}, {conversation_history}) and can be versioned for A/B testing.

Onyx implements hybrid search in Vespa that combines BM25 (sparse, keyword-based) and semantic similarity (dense, embedding-based) scoring. Documents are indexed with both full-text tokens and vector embeddings (768-dim by default), and queries are processed through both pathways with configurable weighting. Results are ranked using a combination of BM25 score and cosine similarity, with optional LLM-based re-ranking for final ordering. The system supports configurable similarity thresholds to filter low-relevance results.

Onyx implements multi-tenancy at the database level with organization-scoped data isolation. Each user belongs to an organization, and all queries are filtered by organization ID at the database layer. Role-based access control (RBAC) is enforced via a permission matrix stored in PostgreSQL, with roles including admin, user, and custom roles. Assistants, documents, and conversations are scoped to organizations, and cross-organization access is prevented at the API server level. Authentication supports SAML, OAuth, and basic auth with session management via JWT tokens.

Onyx uses Celery workers coordinated via Redis to handle long-running tasks asynchronously: document indexing, connector syncs, embedding generation, and LLM inference. Tasks are enqueued with priority levels, and workers process them in parallel. Redis is used for task queue coordination, distributed locking (to prevent duplicate syncs), and caching of frequently-accessed data (embeddings, connector state). The system implements dynamic task scheduling where sync frequency can be adjusted without restarting workers, and failed tasks are retried with exponential backoff.

Onyx stores conversations in PostgreSQL with full message history, including user messages, LLM responses, retrieved documents, and metadata (timestamps, token counts, costs). Each conversation maintains context across turns, enabling multi-turn interactions where the LLM can reference previous messages. The system implements context windowing to manage token limits: older messages are summarized or dropped when conversations exceed the LLM's context window. Conversations are scoped to users and organizations, and can be shared or exported.

Onyx implements configurable document chunking strategies (fixed-size, semantic, recursive) that split documents into retrievable chunks while preserving context. Each chunk is assigned metadata (document ID, source connector, chunk position, document title) for citation tracking. The system supports metadata extraction via LLM-based summarization or rule-based patterns, enabling semantic search on extracted metadata. Chunk size and overlap are configurable per connector, allowing optimization for different document types (code, prose, tables).