agentic-rag-for-dummies

Splits PDF documents into small child chunks (512 tokens) nested within larger parent chunks (2048 tokens), then indexes both layers separately using dense embeddings (sentence-transformers) and sparse BM25 embeddings via FastEmbedSparse. At retrieval time, the system fetches child chunks for precision but returns their parent context for completeness, solving the precision-vs-context tradeoff inherent in flat RAG systems. This two-tier indexing strategy is orchestrated through a DocumentChunker and VectorDatabaseManager that maintains parent-child relationships in Qdrant.

Orchestrates a multi-node LangGraph workflow where an LLM-powered agent reasons about user queries, decides whether to retrieve documents, clarifies ambiguous questions via human-in-the-loop prompts, and iteratively refines search strategies based on retrieval results. The graph implements conditional routing (via graph.add_conditional_edges) to branch between retrieval, clarification, and response generation nodes. State is maintained across turns in a TypedDict that tracks conversation history, retrieved documents, and agent decisions, enabling the agent to learn from previous retrieval failures and adjust its approach.

Processes PDF documents through a multi-stage pipeline: PDF-to-text conversion (with smart routing), hierarchical chunking (parent-child), embedding generation (dense + sparse), and storage in Qdrant. The DocumentManager orchestrates this pipeline, supporting batch indexing of multiple documents and incremental updates (adding new documents without re-indexing existing ones). The pipeline is modular, enabling custom PDF processing strategies or embedding models to be swapped without changing the core indexing logic.

Abstracts vector database operations (insert, search, delete) behind a VectorDatabaseManager class that handles both dense and sparse vector storage in Qdrant. The manager maintains parent-child chunk relationships using Qdrant's metadata filtering, enabling retrieval of child chunks while returning parent context. Supports both in-process (local) and remote Qdrant instances, enabling development on local machines and production on cloud deployments without code changes.

Provides a Jupyter notebook that walks through RAG concepts step-by-step: document loading, chunking, embedding, retrieval, and agent workflows. Each cell is self-contained and executable, enabling learners to understand concepts incrementally and experiment with parameters (chunk sizes, embedding models, LLM providers). The notebook includes visualizations of the indexing pipeline and agent graph, making abstract concepts concrete. This is distinct from the production modular system, serving as an educational tool rather than a deployment artifact.

Implements a dedicated agent node that detects ambiguous or under-specified user queries and generates clarification prompts asking the user to provide additional context (e.g., 'Which department's budget are you asking about?'). The clarification node is triggered via conditional routing when the agent's reasoning indicates insufficient query specificity. User responses are appended to the conversation state and the query is re-processed with the clarified context, enabling iterative refinement without requiring the user to restart the conversation.

Implements three PDF processing strategies (simple text extraction via PyMuPDF4LLM, OCR+table detection for medium-complexity PDFs, and vision-language model analysis for complex layouts) with automatic routing based on PDF characteristics. The DocumentManager analyzes PDF structure (text density, table presence, image complexity) and selects the appropriate strategy, falling back to simpler methods if advanced processing fails. This avoids unnecessary computation (vision models are expensive) while ensuring complex PDFs are handled correctly.

Combines dense vector embeddings (sentence-transformers) and sparse BM25 embeddings (FastEmbedSparse) in a two-stage retrieval pipeline: first, both dense and sparse searches are executed in parallel against Qdrant, then results are merged using reciprocal rank fusion (RRF) to balance semantic relevance and keyword matching. This hybrid approach retrieves child chunks for ranking but returns parent chunks for generation, addressing both semantic gaps (where BM25 fails) and keyword-specific queries (where dense embeddings alone miss exact matches).

Maintains conversation history in the LangGraph state (TypedDict with messages list) across multiple turns, enabling the agent to reference previous queries, clarifications, and retrieved documents when answering new questions. The state includes full message history with roles (user/assistant) and metadata (retrieved documents, agent decisions), allowing the LLM to generate contextually aware responses that acknowledge prior context. Conversation state is passed to every agent node, enabling consistent reasoning across turns without requiring external memory systems.

Defines retrieval tools as Pydantic schemas (e.g., RetrievalTool with query and filters parameters) and exposes them to the LLM via function-calling APIs. The system abstracts provider-specific function-calling implementations (OpenAI, Anthropic, Ollama) behind a unified LangChain interface, enabling the agent to invoke retrieval tools without knowing the underlying LLM provider. Tool schemas include descriptions and parameter constraints, allowing the LLM to understand when and how to use retrieval.

Provides a Gradio-based chat interface that streams agent responses token-by-token to the user, displaying retrieved documents and agent reasoning steps in real-time. The UI integrates with the LangGraph agent, passing user messages to the graph and rendering outputs (responses, clarification prompts, document citations) as they are generated. Streaming is implemented via LangChain's streaming callbacks, reducing perceived latency by showing partial responses while the LLM is still generating.

Centralizes system configuration (LLM provider, model names, chunk sizes, vector database settings) in a configuration module that reads from environment variables and YAML files. This enables switching between Ollama, OpenAI, Anthropic, and Google Gemini by changing a single environment variable (LLM_PROVIDER) without modifying code. Configuration is loaded at startup and passed through the application, enabling runtime provider switching and easy customization for different deployment scenarios (local development vs cloud production).

Uses tiktoken to count tokens in conversation history and automatically prunes old messages when approaching the LLM's context window limit. The system tracks token usage across retrieved documents, conversation history, and system prompts, and removes oldest messages (while preserving recent context) to stay within budget. This enables long conversations without exceeding context limits or requiring manual truncation, though it may lose distant conversation context.