AI21 Labs API

Jamba models combine State Space Models (SSM) with Transformer architecture to achieve 256K context window while maintaining computational efficiency. The hybrid approach uses selective state compression for long-range dependencies and attention mechanisms for precise token interactions, enabling faster inference than pure Transformer models at equivalent context lengths. Requests are processed through AI21's managed inference endpoints with automatic batching and GPU optimization.

API endpoint that accepts a document or text passage and a question, then returns a direct answer grounded in the provided context using the Jamba model's 256K window to maintain document coherence. The system uses attention mechanisms to identify relevant passages and generate answers without hallucinating information outside the provided context. Supports multi-document queries by concatenating inputs within the token limit.

API that analyzes input text and automatically identifies logical segments (paragraphs, sections, chapters, code blocks) and their hierarchical relationships without requiring manual markup. Uses the Jamba model's attention mechanisms to detect structural boundaries based on semantic shifts, formatting patterns, and content coherence. Returns segment boundaries with confidence scores and inferred structure type (heading, body, list, code, etc.).

API endpoint that generates summaries of input text with configurable length targets (e.g., 10%, 25%, 50% of original). Uses Jamba's 256K context to maintain coherence across long documents and applies abstractive techniques (paraphrasing, fusion) rather than extractive selection. Supports multiple summary styles (bullet points, narrative, key facts) and language-aware compression that preserves semantic density.

Service (available via enterprise contract) that enables organizations to fine-tune Jamba models on proprietary datasets to adapt the model for domain-specific tasks, terminology, or style. Fine-tuning uses parameter-efficient techniques (likely LoRA or adapter modules) to avoid full model retraining while maintaining the 256K context capability. Includes evaluation metrics, checkpoint management, and deployment to private endpoints.

Asynchronous batch API that accepts multiple requests (questions, summarization, segmentation tasks) in a single submission and processes them with optimized throughput and reduced per-request latency. Requests are queued, processed in batches on GPU clusters, and results are retrieved via polling or webhook callbacks. Pricing is typically lower per-token than real-time API due to amortized infrastructure costs.

API automatically detects input language and applies language-specific processing (tokenization, segmentation, summarization) without requiring explicit language specification. Jamba models are trained on multilingual data, enabling coherent processing across 50+ languages. Language detection uses lightweight classifiers to identify language before routing to appropriate model variant or processing pipeline.

Utility endpoint that accepts text input and returns the exact token count using Jamba's tokenizer, enabling accurate cost estimation before making API calls. Tokenization uses byte-pair encoding (BPE) with a vocabulary optimized for the Jamba model, ensuring token counts match actual inference costs. Supports batch token counting for multiple inputs in a single request.

API parameter that constrains model outputs to follow a specified JSON schema or format template, enabling extraction of structured data from unstructured text. Uses constrained decoding techniques to enforce schema compliance at token generation time, ensuring outputs are always valid JSON or match specified format. Supports nested objects, arrays, and type validation (string, number, boolean, enum).

API endpoint that returns model outputs as a stream of tokens in real-time using Server-Sent Events (SSE) or WebSocket, enabling applications to display results incrementally as they are generated. Streaming reduces perceived latency by showing partial results immediately rather than waiting for full completion. Supports token-by-token streaming with optional metadata (confidence, logits) for each token.