Cloud Hosted Inference With Usage Based Gpu Time Billing

via “cloud-hosted model inference with usage-based pricing”

Run LLMs locally — simple CLI, model registry, OpenAI-compatible API, automatic GPU detection.

Unique: Meters usage by GPU time rather than tokens, making costs proportional to actual infrastructure utilization. This differs from token-based pricing (OpenAI, Anthropic) and enables context caching to reduce costs for repeated prompts. Cloud models are accessed through the same API as local models, enabling seamless fallback or hybrid local/cloud deployments.

vs others: More cost-predictable than token-based APIs for cache-heavy workloads because context caching reduces GPU time; more flexible than fixed-capacity services because concurrency scales with plan tier; more integrated than separate cloud services because cloud models use the same API as local models.