gptme

Abstracts multiple LLM providers (OpenAI, Anthropic, OpenRouter, local Ollama/llama.cpp) behind a unified provider architecture that normalizes message formats, handles token counting, and manages model-specific capabilities. Uses a provider registry pattern with pluggable backends that transform provider-specific APIs into a common interface, enabling seamless model switching without changing agent logic.

Implements a tool system where LLMs invoke capabilities through a schema-based registry that maps tool names to executable functions. Each tool is a Python class inheriting from a base Tool interface with defined input schemas, execution logic, and output formatting. The agent parses LLM responses for tool invocations, validates against schemas, executes the tool, and feeds results back into the conversation loop.

Provides three separate entry points for agent interaction: a CLI interface (gptme) for terminal use, a REST API server (gptme-server) for programmatic access, and an ncurses UI (gptme-nc) for interactive terminal UI. All interfaces share the same underlying agent logic and tool system, enabling deployment flexibility. The REST API exposes endpoints for chat, tool execution, and conversation management.

Manages conversation state through a message history system that stores all agent-user interactions with metadata (role, timestamp, tool calls). Conversations are persisted to disk (JSON or database) and can be resumed, enabling long-running agents that maintain context across sessions. The system handles message serialization, context window management, and conversation loading/saving.

Generates system prompts dynamically based on agent configuration, available tools, and context. The prompt generation system constructs detailed instructions that describe the agent's role, available tools with their schemas, and execution constraints. Prompts are customizable through configuration files and can be optimized using DSPy for improved agent performance.

Provides an evaluation framework (gptme-eval) that measures agent performance on benchmark tasks using metrics like success rate, token efficiency, and execution time. The framework supports custom evaluation datasets, metric definitions, and comparison across different models and configurations. Results are aggregated and reported with statistical analysis.

Implements a multi-level configuration system where settings can be defined in configuration files (YAML/JSON), environment variables, and command-line arguments, with a clear precedence hierarchy. Configuration is loaded at startup and merged across levels, enabling flexible deployment from development to production without code changes.

Provides a shell tool that executes bash commands in a persistent environment, maintaining working directory state and command history across multiple invocations. Implements safety checks including command whitelisting/blacklisting, output truncation for large results, and error capture with exit codes. Uses subprocess with shell=True but applies filtering rules before execution.

Implements an IPython-based Python execution tool that maintains a persistent Python environment across multiple code executions, enabling stateful computation and variable retention. Code is executed in an isolated namespace with output capture, error handling, and support for imports and library usage. Results are formatted and returned to the agent with execution metadata.

Provides three complementary file tools: save (create new files), patch (apply unified diff format changes), and append (add content to existing files). The patch tool parses git-style diffs and applies them incrementally, enabling surgical edits without rewriting entire files. All operations include error handling and validation to prevent data loss.

Integrates Playwright for browser automation, enabling the agent to navigate websites, interact with elements, extract content, and capture screenshots. The browser tool manages a persistent browser session with support for JavaScript execution, form filling, and dynamic content loading. Results are returned as text extracts or screenshots for agent analysis.

Provides vision capabilities through a vision tool that analyzes images using multimodal LLM models (Claude Vision, GPT-4V), and a screenshot tool that captures desktop/window screenshots. Images are encoded as base64 and sent to the LLM with natural language queries, returning structured analysis. Screenshot tool integrates with the system to capture current state for agent awareness.

Implements a subagent tool that enables agents to spawn child agents for specific subtasks, creating a hierarchical execution model. Child agents inherit the parent's configuration and tools but operate independently with their own conversation history. Results are aggregated and returned to the parent agent, enabling complex multi-step workflows with task isolation.

Provides a tmux tool that manages long-running background processes through tmux sessions, enabling the agent to start processes, monitor their status, and retrieve output without blocking. Processes run in isolated tmux sessions that persist across agent invocations, allowing the agent to check status and collect results asynchronously.

Implements a RAG tool that indexes documents (code files, markdown, PDFs) into a vector database using embeddings, enabling semantic search over large codebases or knowledge bases. The tool supports querying with natural language, returning relevant documents ranked by semantic similarity. Uses embeddings from the configured LLM provider or local models.