AI21 Jamba 1.5

Processes up to 256K tokens using a hybrid architecture that interleaves Mamba structured state space layers (providing linear-time sequence processing) with Transformer attention layers (providing precise token interactions). The Mamba layers enable efficient memory usage and fast inference on long sequences by maintaining a compact state representation, while Transformer layers preserve fine-grained attention patterns where needed. This dual-layer approach allows the model to handle massive documents and multi-document reasoning tasks without the quadratic memory overhead of pure Transformer architectures.

Provides instruction-tuned and chat-optimized model variants (Jamba 1.5 Instruct and Jamba 1.5 Chat) that follow user directives, answer questions, engage in multi-turn conversations, and complete general language tasks. The models are fine-tuned using standard instruction-following and RLHF-style techniques (methodology not publicly detailed) to align with user intent and maintain conversational coherence across multiple exchanges.

Jamba models are released as open-source with weights available on Hugging Face, enabling community contributions, research, and custom deployments. The open-source approach allows researchers to study the hybrid Mamba-Transformer architecture, contribute improvements, and build upon the models. Community members can create optimized inference implementations, fine-tuning guides, and domain-specific adaptations without licensing restrictions.

Leverages the 256K context window to simultaneously process multiple documents and perform reasoning across them, identifying relationships, contradictions, and synthesizing information without requiring external retrieval or document ranking. The model can ingest entire document sets (e.g., multiple research papers, financial reports, contracts) in a single forward pass and generate coherent summaries, comparisons, or analyses that reference specific sections across all input documents.

The Mamba state space layers provide linear-time sequence processing (O(n) complexity vs. O(n²) for Transformer attention), enabling faster inference and lower GPU memory consumption compared to pure Transformer models of similar capability. The model maintains a compact hidden state representation that doesn't require storing full attention matrices, reducing peak memory usage during inference and enabling deployment on smaller GPUs or edge devices.

Provides hosted inference through AI21 Studio API with transparent per-token pricing for input and output tokens. Users submit text requests via REST API and receive responses with token usage tracking, enabling cost-predictable inference without managing infrastructure. Pricing varies by model variant (Mini at $0.2/$0.4 per 1M input/output tokens, Large at $2/$8 per 1M tokens) and includes free trial credits ($10 for 3 months).

Models are available for download from Hugging Face in standard formats (likely safetensors or PyTorch), enabling self-hosted deployment on custom infrastructure. Users can run Jamba locally on their own GPUs, integrate with inference frameworks (vLLM, TensorRT, Ollama), and maintain full control over data, inference latency, and scaling. This approach eliminates API latency and per-token costs but requires infrastructure management and optimization expertise.

Jamba models can be fine-tuned on custom datasets to adapt to specific domains, tasks, or writing styles. While the fine-tuning methodology is not publicly documented, the hybrid architecture suggests compatibility with standard fine-tuning approaches (full fine-tuning, LoRA, QLoRA). Fine-tuning leverages the model's instruction-following foundation and adapts the Mamba-Transformer hybrid to domain-specific patterns, enabling specialized performance without training from scratch.

Designed for enterprise use cases involving large-scale document processing, knowledge base search, and information extraction from structured and unstructured documents. The 256K context window enables processing of entire documents without chunking, and the efficient inference enables cost-effective batch processing of large document collections. Supports integration with enterprise knowledge management systems, document repositories, and compliance workflows.

AI21 claims that Jamba achieves up to 30% more text per token compared to other providers, suggesting optimized tokenization or more efficient token usage. This efficiency reduces the number of tokens required to represent the same amount of text, directly lowering API costs and enabling more content to fit within the 256K context window. The specific tokenization approach (vocabulary size, encoding scheme) is not documented, but the efficiency claim suggests careful vocabulary design or subword tokenization optimization.

The Chat variant of Jamba maintains conversation state across multiple turns, enabling coherent multi-turn dialogues where the model tracks context, user preferences, and conversation history. The 256K context window allows storing extensive conversation history without truncation, enabling long-running conversations with full context awareness. The model can reference earlier exchanges, maintain consistent personas, and adapt responses based on accumulated conversation context.