GPT Runner

Enables multi-turn dialogue with an LLM agent that maintains context of local files and directories, allowing developers to ask questions about code structure, logic, and relationships without manually copying content into prompts. The agent indexes file paths and content, tracks conversation history, and routes queries to the appropriate files based on semantic understanding of developer intent.

Processes multiple files in sequence through an LLM pipeline, applying consistent transformations, analyses, or generations across a codebase. The agent reads each file, sends it to the LLM with a specified prompt template, and writes results back to the filesystem or collects them for review, enabling bulk code refactoring, documentation generation, or linting-style operations at scale.

Automatically saves multi-turn conversations with file context to disk, allowing developers to pause analysis and resume later without losing dialogue history or re-establishing context. The agent serializes conversation state (messages, file references, LLM responses) to a structured format and reconstructs the full context when a session is reopened, maintaining semantic continuity across sessions.

Provides a unified API for interacting with multiple LLM providers (OpenAI, Anthropic, local models via Ollama, etc.) without changing application code. The agent abstracts provider-specific API differences (authentication, request/response formats, parameter names) behind a common interface, allowing developers to swap providers or use multiple providers in parallel by changing configuration.

Streams LLM responses token-by-token to the user interface or console as they are generated, rather than waiting for the complete response. The agent pipes the LLM's streaming output directly to the output stream, providing immediate feedback and reducing perceived latency for long-running analyses or code generation tasks.

Provides a templating engine for defining reusable prompts with placeholders for dynamic values (file paths, code snippets, user queries). The agent substitutes variables at runtime before sending prompts to the LLM, enabling consistent prompt engineering across multiple queries and batch operations without hardcoding values.

Builds an in-memory or persistent index of file contents, enabling semantic search queries to find relevant files or code snippets without reading the entire filesystem. The agent may use keyword matching, embeddings, or AST-based indexing to quickly locate files matching developer queries, reducing the context needed for each LLM call.

Detects LLM errors, API failures, and malformed outputs, then provides actionable guidance to users on how to resolve issues. The agent may suggest retrying with different parameters, checking API credentials, or reformulating queries, and can automatically retry transient failures with exponential backoff.

Executes code generated or suggested by the LLM in a controlled environment, validating that it runs without errors before presenting it to the user. The agent may use subprocess isolation, Docker containers, or language-specific sandboxes to safely execute code and capture output, preventing malicious or broken code from affecting the developer's system.

Manages configuration for LLM providers, file paths, prompt templates, and other settings through configuration files or environment variables, allowing developers to adapt the agent's behavior without code changes. The agent loads configuration at startup and applies environment-specific overrides (development, staging, production).