PrivateGPT

Accepts documents in multiple formats (PDF, DOCX, TXT, etc.), automatically parses and splits them into semantically meaningful chunks using configurable chunk size and overlap parameters, then embeds each chunk using a pluggable embedding model (local or cloud-based). The ingestion pipeline stores both embeddings in a vector database and raw chunk text/metadata in a node store for later retrieval and context assembly.

Implements a full RAG pipeline that embeds user queries, retrieves semantically similar chunks from the vector store, optionally reranks retrieved results for relevance, and assembles retrieved context into a prompt template before sending to an LLM. The pipeline supports both synchronous and streaming responses, with configurable retrieval parameters (top-k, similarity threshold) and optional reranking models to improve answer quality.

Maintains conversation history across multiple turns, allowing users to ask follow-up questions that reference previous answers. The system assembles context from both the current query and relevant previous turns, passes this to the LLM for coherent multi-turn responses. Chat history is stored in memory (or optionally persisted) and can be cleared or managed per conversation session.

Extracts and stores metadata from documents (filename, upload date, document type, custom tags) alongside embeddings, enabling metadata-based filtering during retrieval. Users can filter search results by metadata (e.g., 'only search in PDFs from 2024') to improve precision. Metadata is stored in the node store and can be used in hybrid search combining semantic similarity with keyword/metadata filtering.

Supports batch ingestion of multiple documents through an asynchronous pipeline that processes documents in parallel without blocking the API. Documents are queued, processed by worker threads/processes, and their ingestion status can be monitored via API endpoints. This enables efficient ingestion of large document collections without blocking the main application.

Provides a templating system for assembling prompts that combine user queries, retrieved context, and system instructions. Developers can customize prompt templates via YAML configuration to control how context is formatted, what instructions are given to the LLM, and how responses are structured. Supports variable substitution (e.g., {query}, {context}, {date}) and conditional sections based on available context.

Abstracts LLM interactions through LlamaIndex's LLM interface, supporting local models (via Ollama), OpenAI, Anthropic, Hugging Face, and other providers through a unified configuration layer. Developers specify the LLM provider in YAML config without code changes, and the system handles API authentication, request formatting, and response parsing for each provider's unique protocol.

Abstracts vector storage through LlamaIndex's vector store interface, supporting Qdrant, Milvus, Weaviate, Pinecone, and in-memory SimpleVectorStore. Developers configure the vector store backend in YAML, and the system handles connection pooling, index creation, similarity search, and metadata filtering without code changes. Supports both dense vector search and hybrid search (combining vector similarity with keyword matching).

Provides a dedicated summarization service that generates summaries of ingested documents using the configured LLM. Supports multiple summarization strategies (e.g., map-reduce for long documents, refine for iterative improvement) and can summarize individual documents or entire collections. Summaries are cached and can be retrieved alongside search results to provide high-level overviews before diving into detailed chunks.

Uses a dependency injection (DI) pattern to decouple all major components (LLM, embedding model, vector store, retriever, reranker) from the application logic. Components are registered in a container and injected into services at runtime, allowing developers to swap implementations without modifying service code. This enables easy testing, custom component implementations, and runtime configuration changes.

Provides a centralized YAML configuration system that controls all aspects of PrivateGPT (LLM provider, embedding model, vector store, chunking strategy, API settings) without requiring code changes. Supports environment variable substitution for sensitive values (API keys, connection strings) and multiple configuration profiles (dev, staging, production) for different deployment environments.

Exposes PrivateGPT functionality through a FastAPI REST API with both synchronous endpoints (for simple requests) and streaming endpoints (for long responses). The API supports document ingestion, chat/Q&A, summarization, and document listing operations. Streaming endpoints use Server-Sent Events (SSE) to send response tokens incrementally, enabling real-time UI updates and better perceived performance.

Provides a built-in Gradio web interface for non-technical users to upload documents, ask questions, and view answers without writing code. The UI supports drag-and-drop document upload, displays retrieved source chunks alongside answers, and provides a chat-like interface for multi-turn conversations. The UI is fully optional — developers can build custom UIs using the REST API instead.

Implements a privacy-first architecture where all processing (document parsing, embedding, retrieval, LLM inference) happens locally by default — no data is sent to external services unless explicitly configured. Supports optional integration with cloud LLM providers (OpenAI, Anthropic) for cases where local models are insufficient, but this is opt-in and configurable per deployment. Developers can choose to run entirely on-premise with local models (Ollama) or hybrid (local embedding + cloud LLM).