together

Provides both synchronous (Together) and asynchronous (AsyncTogether) HTTP clients built on httpx with configurable exponential backoff retry strategies for transient failures. The architecture uses a base client pattern (_BaseClient) that abstracts HTTP operations, allowing runtime selection between httpx (default) and aiohttp backends for async workloads. Automatic retry logic with configurable max retries and backoff multipliers handles network transience without developer intervention.

Implements real-time token streaming via Server-Sent Events (SSE) for both synchronous and asynchronous clients by setting stream=True on API calls. The streaming layer (_streaming.py) parses SSE-formatted responses and yields individual tokens or completion chunks as they arrive from the server, enabling low-latency token consumption for chat and text generation endpoints. Supports both line-by-line iteration (sync) and async iteration patterns.

Implements the batch resource for processing large numbers of requests asynchronously in a single batch job. Developers submit a JSONL file containing multiple API requests, and the batch API processes them in parallel, returning results in a JSONL output file. Batch processing is significantly cheaper than real-time API calls but introduces latency (typically hours). The API provides job status monitoring and result retrieval.

Implements the files resource for managing data files used in fine-tuning, batch processing, and other workflows. The API provides file.upload (with format validation), file.retrieve (download), file.list (enumerate), and file.delete operations. Files are stored on Together's servers and referenced by file_id in downstream operations. The API validates file format (JSONL for training data) and provides storage quotas.

Implements the models resource for discovering available models and retrieving their metadata (context window, pricing, capabilities, etc.). The API provides models.list() to enumerate all available models and models.retrieve(model_id) to get detailed information about a specific model. Model metadata includes supported features (chat, completions, embeddings, etc.), pricing, and availability status.

Provides command-line interface (CLI) tools for managing files, models, fine-tuning jobs, and clusters without writing Python code. The CLI mirrors the SDK API surface, exposing commands like 'together files upload', 'together fine-tuning create', 'together models list', etc. CLI tools are useful for scripting, automation, and interactive exploration of the Together API.

Implements a comprehensive error handling system with typed exception classes (APIError, AuthenticationError, RateLimitError, etc.) that provide context about failures. The SDK automatically retries transient errors (5xx, timeouts) with exponential backoff, but raises typed exceptions for application-level errors (4xx, auth failures). Error objects include request_id for debugging and suggestions for recovery.

Provides a fully asynchronous client (AsyncTogether) that mirrors the synchronous Together client but uses async/await syntax and integrates with Python's asyncio event loop. All API resources are available on the async client with identical signatures. The async client uses aiohttp (optional) or httpx for HTTP operations, enabling high-concurrency workloads without blocking threads.

Organizes 15+ API resources (chat, completions, images, audio, embeddings, fine-tuning, etc.) as typed attributes on the client using Pydantic models for response validation and TypedDict for request parameters. The type system enforces schema validation at runtime via Pydantic, catching malformed requests before they reach the API. Resources are lazily loaded via the _resources_proxy pattern to minimize import overhead.

Implements the chat.completions resource that accepts a list of Message objects (with role, content, and optional tool_calls) and returns a ChatCompletion response with choice objects containing the assistant's reply. Supports system prompts, multi-turn conversation history, and tool/function calling via the tools parameter. The API maintains conversation context across multiple calls, allowing developers to build stateful chat applications by managing message history client-side.

Implements the completions resource for raw text generation (non-chat) that accepts a prompt string and returns a Completion response with generated text. Supports sampling parameters (temperature, top_p, top_k, repetition_penalty) for controlling output diversity and quality. Unlike chat completions, this endpoint does not maintain conversation context and is optimized for single-turn prompt-based generation tasks.

Implements the images.generate resource that accepts a text prompt and returns generated images as URLs or base64-encoded data. Supports model selection (DALL-E 3, Stable Diffusion variants), quality parameters (steps, guidance_scale, seed), and output format control (url or base64). The API abstracts differences between underlying image generation models, providing a unified interface.

Implements audio resources for transcription (speech-to-text via audio.transcriptions) and synthesis (text-to-speech via audio.speech). The transcription endpoint accepts audio files (WAV, MP3, M4A) and returns transcribed text with optional language detection. The speech endpoint accepts text and returns audio in specified format (MP3, WAV, etc.). Both endpoints support model selection and quality parameters.

Implements the embeddings.create resource that accepts text inputs and returns dense vector embeddings for semantic search, clustering, or similarity comparison. Supports batch processing of multiple texts in a single request, model selection (e.g., 'BAAI/bge-large-en-v1.5'), and configurable embedding dimensions. The API returns Embedding objects with vector data and input text reference.

Implements the fine_tuning resource for training custom models on user-provided datasets. The API manages the full fine-tuning lifecycle: dataset upload (via files.upload), job creation (fine_tuning.jobs.create), status monitoring (fine_tuning.jobs.retrieve), and model deployment. Fine-tuning jobs are asynchronous and can be polled for completion status. The API validates dataset format and provides training metrics.

Implements the endpoints resource for deploying fine-tuned or custom models to dedicated inference endpoints with guaranteed availability and performance. The API manages endpoint lifecycle: creation (endpoints.create), status monitoring (endpoints.retrieve), and inference (endpoints.chat.completions). Dedicated endpoints provide lower latency and higher throughput than shared API endpoints, with optional auto-scaling.