Gitingest

Walks the Git repository's file tree structure, respects .gitignore rules to filter out non-essential files, and aggregates source code and documentation into a single unified text document. Uses Git APIs or filesystem traversal to enumerate files while applying ignore patterns, then concatenates file contents with metadata markers (file paths, line counts) to preserve structure for LLM consumption.

Clones or fetches Git repositories from remote sources (GitHub, GitLab, Gitea, Gitee, etc.) without requiring users to pre-clone locally. Supports shallow cloning (single branch, limited history) to minimize bandwidth and latency for large repositories. Uses Git CLI or libgit2 bindings to authenticate and fetch repository metadata and content.

Allows users to define custom filtering rules beyond .gitignore (e.g., include only Python files, exclude files larger than 1MB, exclude test directories) via UI options, API parameters, or configuration files. Applies filters in addition to or instead of .gitignore rules, enabling fine-grained control over digest content.

Parses and applies .gitignore rules to exclude files from the digest, using pattern matching (wildcards, negations, directory-specific rules) consistent with Git's own ignore semantics. Implements gitignore spec compliance to avoid including build artifacts, node_modules, .env files, and other non-essential content that would bloat the LLM context.

Detects file types by extension and applies language-specific formatting (indentation, line breaks, comment markers) when aggregating code into the digest. Preserves syntax structure and readability for LLMs by maintaining code formatting, adding file path headers, and optionally including line numbers. Does not perform parsing or AST analysis — purely structural formatting for readability.

Estimates the token count of the generated digest using language model-specific tokenizers (e.g., tiktoken for OpenAI models) and provides warnings or truncation suggestions when the digest exceeds typical LLM context windows (4k, 8k, 16k, 128k tokens). May offer compression strategies (file filtering, summarization hints) to fit within token budgets.

Processes multiple Git repositories in parallel or batch mode, generating digests for each and optionally combining them into a single multi-repository document. Uses concurrent fetching and processing to reduce total execution time compared to sequential ingestion. May support batch input formats (CSV, JSON) listing repository URLs.

Provides a web interface where users can paste or search for Git repository URLs, configure filtering options (file types, size limits, .gitignore respect), preview the generated digest, and download or copy it for LLM use. Offers real-time feedback on digest size, token count, and file inclusion decisions.

Exposes a REST or GraphQL API that accepts repository URLs and configuration parameters, returns generated digests in JSON or plain text format, and supports webhooks or async processing for large repositories. Enables integration with external tools, CI/CD systems, and LLM workflows without requiring direct web UI interaction.

Generates digests in multiple output formats including plain text, Markdown with code blocks and headers, JSON with file metadata, and optionally YAML or CSV for structured analysis. Markdown output includes table of contents, file structure trees, and section headers for better organization and readability in documentation tools.

Allows users to specify a Git branch, tag, or commit hash to generate digests from specific points in repository history. Enables comparison of codebases across versions, analysis of historical code patterns, or ingestion of stable releases rather than development branches. Fetches the specified ref without requiring full history download.