Dify

Dify implements a node factory pattern with dependency injection to execute directed acyclic graphs (DAGs) where each node type (LLM, HTTP, code, knowledge retrieval, human input) is instantiated and executed in dependency order. The workflow engine manages state transitions, pause-resume mechanics via human input nodes, and error handling across multi-step pipelines. Nodes are defined declaratively in JSON/YAML and compiled into executable graphs at runtime.

Dify implements a pluggable RAG system with a vector database factory pattern that abstracts over multiple backends (Weaviate, Pinecone, Milvus, Qdrant, etc.). The retrieval pipeline supports multiple strategies: dense vector similarity, BM25 hybrid search, metadata filtering, and summary index generation. Documents are chunked, embedded, and indexed asynchronously via Celery background tasks. The knowledge retrieval node in workflows can be configured with custom retrieval parameters and re-ranking strategies.

Dify instruments the entire application stack with OpenTelemetry (OTEL) for distributed tracing, metrics collection, and logging. Traces capture request flow through the API, workflow execution, LLM calls, and database queries. The system integrates with Sentry for error tracking and performance monitoring. Metrics include request latency, token usage, error rates, and queue depth. Logs are structured (JSON) and include trace context for correlation. The observability system is configurable to send data to external collectors (Jaeger, Datadog, etc.).

Dify supports integrating external knowledge bases via API calls, enabling workflows to retrieve information from systems outside Dify (e.g., Confluence, Notion, custom databases). The knowledge retrieval node can be configured to call external APIs instead of querying local vector databases. The system handles API authentication, response parsing, and result ranking. External knowledge bases are treated as first-class citizens alongside local datasets, allowing seamless switching between local and external sources.

Dify provides an annotation interface where users can review workflow outputs, provide feedback (correct/incorrect, ratings, comments), and curate datasets. Annotations are stored with context (input, output, feedback, annotator) and can be exported for model fine-tuning or evaluation. The system supports batch annotation workflows and annotation templates for consistent feedback. Annotations are tracked with versioning, allowing rollback if needed. The annotation data feeds into model evaluation pipelines.

Dify supports versioning of applications (workflows, prompts, datasets) with automatic version tracking on each save. Applications can be deployed to different environments (development, staging, production) with environment-specific configurations (API keys, model selections, parameters). The system tracks deployment history and allows rollback to previous versions. Applications can be published as public APIs or embedded in websites. Version comparison shows changes between versions, enabling easy review of modifications.

Dify implements a provider and model architecture that abstracts over 20+ LLM providers (OpenAI, Anthropic, Ollama, Azure, etc.) through a unified invocation pipeline. The system manages API keys per provider, enforces quota limits via credit pools, tracks token usage per model, and supports streaming responses. Model invocation is instrumented with OpenTelemetry for observability. The architecture uses a provider registry pattern to dynamically load provider implementations at runtime.

Dify integrates the Model Context Protocol (MCP) to enable external tools and services to be plugged into workflows via a standardized interface. The system runs a plugin daemon that manages MCP server lifecycle, handles tool discovery, and executes tool calls with sandboxed environments. Tools can be built-in (HTTP requests, code execution), API-based (external services), or MCP-compliant servers. The tool provider architecture uses a factory pattern to instantiate different tool types and manage their execution context.

Dify provides a dataset service that manages document lifecycle: upload, parsing, chunking, embedding, and indexing. Documents are parsed based on file type (PDF, DOCX, TXT, etc.), split using recursive character splitting with configurable chunk size and overlap, embedded using configurable embedding models, and indexed asynchronously via Celery. The system tracks document metadata (source, upload date, processing status) and supports incremental updates. Datasets can be used directly in knowledge retrieval nodes or exported as external knowledge bases.

Dify provides a visual prompt editor that supports variable interpolation (using {{variable}} syntax), prompt templates with system/user/assistant roles, and built-in testing against multiple LLM providers. The IDE includes prompt versioning, A/B testing capabilities, and performance metrics (latency, token usage, cost). Prompts can be parameterized with input variables that are bound at runtime from workflow context or API parameters. The system tracks prompt history and allows rollback to previous versions.

Dify implements a multi-tenant architecture where each workspace is a logical isolation boundary with separate datasets, workflows, API keys, and member permissions. The system uses role-based access control (RBAC) with predefined roles (admin, editor, member, guest) that control access to resources. Tenant isolation is enforced at the database query level using tenant context middleware. Authentication supports multiple methods: email/password, OAuth, SAML, and API keys. Member management includes invitations, role assignment, and audit logging.

Dify exposes a streaming chat API that accepts user messages, maintains conversation history, and returns LLM responses as server-sent events (SSE) for real-time streaming. The API supports multi-turn conversations with automatic context management (previous messages are included in the prompt). Users can provide feedback on responses (thumbs up/down, ratings, comments) which is stored for model improvement. The system tracks conversation metadata (start time, duration, model used, tokens consumed) and supports conversation export.

Dify exposes a workflow execution API that accepts workflow input parameters, queues the workflow for execution via Celery, and returns a job ID for polling results. Workflows execute asynchronously in background workers, enabling long-running operations without blocking the API. The system supports streaming results via SSE for real-time progress updates. Execution traces are captured with node-level logs, token usage, and latency metrics. Results can be polled via the job ID or streamed via WebSocket.

Dify provides file upload endpoints that accept documents, images, and other files, validate file types and sizes, scan for viruses using ClamAV, and store files in configurable backends (local filesystem, S3, etc.). Uploaded files are tracked with metadata (filename, size, upload time, user) and can be referenced in workflows and datasets. The system supports resumable uploads for large files and automatic cleanup of temporary files. Files are served with access control to prevent unauthorized downloads.