llm-spend-guard

Intercepts and monitors token usage in real-time by wrapping API calls to OpenAI, Anthropic Claude, and Google Gemini, tracking input/output tokens per request and maintaining cumulative counters. Uses provider-specific token counting libraries (tiktoken for OpenAI, custom counters for Anthropic/Gemini) to calculate costs before responses are returned, enabling immediate visibility into consumption patterns without post-hoc analysis.

Validates incoming requests against configurable per-request token budgets before sending to LLM APIs, rejecting calls that would exceed limits and throwing typed errors. Implements budget checking by calculating estimated input tokens from the request payload and comparing against a configured threshold, preventing over-budget requests from reaching the API and incurring charges.

Tracks total token spending across all requests within a session or time window and enforces a cumulative budget ceiling, rejecting new requests when the session total would exceed the configured limit. Maintains an in-memory accumulator of costs per session, comparing each new request's estimated cost against remaining budget and blocking requests that would push the session over the threshold.

Converts token counts to USD costs using provider-specific pricing tables (OpenAI GPT-4/GPT-4o, Anthropic Claude variants, Google Gemini tiers), normalizing costs across providers into a single currency for comparison and aggregation. Implements a pricing registry that maps model names to per-token input/output rates, calculating costs as (input_tokens × input_rate) + (output_tokens × output_rate) and supporting multiple model variants per provider.

Wraps LLM API calls (OpenAI, Anthropic, Google Gemini) with a unified interface that transparently injects token counts and cost data into responses without modifying the underlying API contract. Uses middleware/decorator pattern to intercept requests before sending to providers and responses after receiving, enriching response objects with usage metadata (tokens, cost) while preserving the original provider response structure.

Monitors spending patterns and triggers alerts when costs exceed configured thresholds (per-request, per-session, or per-time-window), enabling proactive detection of budget overruns or unexpected usage spikes. Implements threshold comparison logic that evaluates current spending against configured limits and emits events or callbacks when thresholds are crossed, supporting multiple alert levels (warning, critical) and custom handlers.

Manages budget reset schedules (daily, weekly, monthly) and time-window-based quota enforcement, automatically resetting cumulative spending counters at configured intervals and supporting sliding-window or fixed-window quota models. Implements timer-based reset logic that clears session budgets or resets global counters at specified times, enabling per-period spending limits without manual intervention.

Records comprehensive logs of all API calls, token usage, costs, and budget decisions (approvals/rejections) with timestamps and context, enabling audit trails and usage analytics. Implements structured logging that captures request metadata (model, user, session), token counts (input/output), costs, and budget enforcement decisions, supporting multiple log destinations (console, file, external services) via configurable handlers.

Provides typed error objects and recovery strategies when budgets are exhausted, including graceful degradation options (fallback models, request truncation, queuing) and error callbacks for custom handling. Implements error classification (budget exceeded, invalid model, API error) with structured error objects that include remaining budget, suggested actions, and recovery hints.