FastGPT

FastGPT provides a drag-and-drop workflow editor that compiles visual node graphs into a directed acyclic graph (DAG) executed server-side with streaming support. The system resolves variable dependencies across nodes, supports branching logic, pause-resume semantics for interactive workflows, and child workflow composition. Each node type (AI, HTTP, dataset query, etc.) has a standardized execution interface that handles both synchronous and asynchronous operations with real-time streaming of intermediate results back to the client.

FastGPT abstracts LLM provider APIs (OpenAI, Anthropic, Qwen, DeepSeek, Ollama, etc.) behind a unified request interface that handles model selection, streaming response aggregation, token counting, and cost tracking. The system normalizes chat message formats across providers, manages API key rotation, implements retry logic with exponential backoff, and streams partial responses to clients in real-time. Token usage is tracked per request and aggregated for billing/analytics.

FastGPT provides Docker images and Kubernetes manifests (Helm charts) for containerized deployment, with comprehensive environment variable configuration for all components (backend, frontend, vector DB, etc.). The system includes health checks, resource limits, and scaling policies. Deployment documentation covers single-container setups, multi-replica production deployments, and cloud-specific configurations (AWS, GCP, Azure). Environment variables control feature flags, database connections, and LLM provider credentials.

FastGPT includes an observability SDK that collects structured logs, traces, and metrics from all components (workflows, LLM calls, database operations, etc.). The system integrates with popular observability platforms (Datadog, New Relic, Prometheus) via standard protocols (OpenTelemetry). Logs include request IDs for tracing across services, structured fields for filtering/searching, and configurable log levels. Metrics cover latency, error rates, token usage, and cost tracking.

FastGPT provides a testing framework that allows users to create test cases for workflows, run them against different model configurations, and track metrics like accuracy, latency, and cost. The system supports batch testing with result comparison, A/B testing between workflow versions, and metric aggregation across test runs. Test results are stored with full execution logs for debugging. The framework integrates with the workflow editor for easy test creation and execution.

FastGPT supports publishing workflows as reusable plugins that can be shared with other users or teams via a built-in marketplace. Plugins can be simple workflows or complex tools with custom UI. The system handles plugin versioning, dependency management, and installation. Users can browse available plugins, install them with one click, and customize them for their use case. Plugin authors can monetize their work via the marketplace.

FastGPT implements a multi-stage retrieval pipeline that converts documents into embeddings, stores them in vector databases, and retrieves relevant chunks via semantic similarity search combined with BM25 keyword matching. The system supports hierarchical dataset organization, configurable chunk size and overlap, multiple embedding models, and re-ranking of results before passing to LLMs. Retrieved context is automatically injected into chat prompts with source attribution and confidence scores.

FastGPT provides a data pipeline that ingests documents in multiple formats (PDF, DOCX, TXT, Markdown, JSON, CSV), automatically chunks them with configurable size/overlap, generates embeddings, and stores chunks in vector databases with metadata. The system supports incremental updates (add/delete chunks without re-processing entire dataset), batch processing with progress tracking, and automatic format detection. Chunks are versioned and linked to source documents for traceability.

FastGPT implements MCP server support that allows workflows to discover and execute tools from external MCP servers via a standardized protocol. The system maintains a tool registry, handles tool schema validation, manages tool execution with timeout/error handling, and streams tool results back into workflow execution. Tools can be published as plugins and shared across teams. The MCP integration layer abstracts provider differences (Anthropic, custom implementations) and handles authentication/authorization.

FastGPT provides a chat UI component that streams LLM responses in real-time, supports variable input forms for workflow initialization, maintains conversation history with context management, and handles both shared (public) and authenticated chat modes. The chat system manages message state, implements pagination for long conversations, supports file uploads in chat, and provides feedback mechanisms (thumbs up/down) for response quality tracking. Responses can be streamed token-by-token or chunk-by-chunk depending on workflow configuration.

FastGPT implements a fine-grained permission system with role-based access control (RBAC) supporting team hierarchies, resource-level permissions, and permission inheritance. Users can be assigned roles (owner, editor, viewer) at team or resource level, with permissions cascading from parent to child resources. The system supports sharing resources via public links with optional expiration, and maintains audit logs of permission changes. API keys can be scoped to specific resources and operations.

FastGPT exposes deployed applications via REST APIs compatible with OpenAI's chat completion format, allowing applications to be consumed by any client supporting that standard. The system generates API keys scoped to specific applications, implements rate limiting and quota management, provides webhook support for async operations, and includes built-in API documentation. Applications can be deployed as simple chatbots or complex workflows, and the API layer handles request routing, authentication, and response formatting automatically.

FastGPT maintains version history for workflows, allowing users to view changes between versions, rollback to previous versions, and compare workflow definitions. The system tracks who made changes and when, stores complete workflow snapshots, and supports branching workflows from specific versions. Version management is integrated into the workflow editor UI with visual diff support. Rollbacks are instant and don't affect running workflow instances.

FastGPT abstracts vector database implementations behind a unified interface, supporting Milvus, Weaviate, Qdrant, PineCone, and Chroma. The system handles database-specific connection details, query syntax translation, and schema management transparently. Users can switch vector databases without changing application code. The platform also supports multiple relational databases (PostgreSQL, MySQL) for metadata storage, with automatic migration scripts for version upgrades.