GooseAi

Provides HTTP-based access to multiple language models (125M to 20B parameters) with per-token billing and competitive pricing undercut to OpenAI's GPT-3.5. Uses standard REST endpoints for prompt submission and streaming or batch response retrieval, with request/response payloads structured as JSON. The pricing model charges only for tokens consumed, enabling fine-grained cost control for production inference workloads at scale.

Exposes a range of model sizes from 125M to 20B parameters as selectable endpoints, allowing developers to choose inference speed vs. output quality based on workload requirements. The API accepts a 'model' parameter in requests to route to different model variants. Smaller models (125M-1B) prioritize latency for real-time applications, while larger models (7B-20B) improve coherence and reasoning at the cost of higher latency and per-token cost.

Provides a Python library that mirrors OpenAI's client interface, allowing developers to swap API endpoints with minimal code changes. The SDK handles HTTP request serialization, response parsing, error handling, and retry logic internally. It supports both synchronous and asynchronous (async/await) patterns, with context managers for resource cleanup. The compatibility layer maps GooseAI model names and parameters to OpenAI's expected format, reducing cognitive load for teams familiar with OpenAI's SDK.

Tracks and reports token consumption at the request level, returning detailed usage metadata (prompt tokens, completion tokens, total tokens) in API responses. This enables developers to calculate per-request costs using published per-token rates and attribute spending to specific features, users, or workloads. The SDK and REST API both expose usage information in response objects, allowing integration with cost monitoring and billing systems.

Supports submitting multiple inference requests as a batch job for asynchronous processing, allowing developers to trade latency for throughput and cost savings. Batch jobs are queued and processed during off-peak hours, typically returning results within hours rather than milliseconds. The API returns a job ID for polling or webhook-based result retrieval, enabling developers to decouple request submission from result consumption.

Exposes standard LLM sampling parameters (temperature, top_p, top_k, frequency_penalty, presence_penalty) in the API, allowing developers to control output randomness and diversity. Temperature scales logits before sampling (0 = deterministic, 1+ = more random), while top_p and top_k implement nucleus and top-k sampling respectively. These parameters are passed per-request, enabling dynamic control over model behavior without retraining or fine-tuning.

Offers a free account tier with monthly token allowances (typically 5,000-10,000 free tokens) and rate limits, enabling developers to experiment and prototype without upfront payment. Free tier accounts have reduced rate limits (e.g., 10 requests/minute) and may have access to smaller models only. Upgrading to paid accounts removes rate limits and provides higher monthly allowances with pay-as-you-go billing.