Mistral API

Provides access to a tiered model family (Mistral Large, Medium, Small) through a unified API endpoint, allowing developers to select models based on latency/cost/capability tradeoffs. Each model is optimized for parameter efficiency, with routing logic that maps requests to the appropriate model tier. The API handles tokenization, context windowing, and response streaming through standard HTTP/gRPC interfaces with configurable temperature, top-p, and max-tokens parameters.

Enforces JSON schema compliance in model outputs by constraining the token generation process to only produce valid JSON matching a developer-provided schema. The implementation uses grammar-based token masking during decoding — at each generation step, only tokens that maintain JSON validity are allowed, preventing malformed output. Schemas are specified as JSON Schema Draft 7 objects passed in the API request, and the model guarantees output will parse without errors.

Generates dense vector embeddings from text that capture semantic meaning, enabling similarity search, clustering, and retrieval-augmented generation (RAG). The API accepts text inputs and returns fixed-dimensional vectors (typically 1024 or 4096 dimensions depending on model) that can be stored in vector databases. Supports batch embedding generation for efficiency and includes normalization options for different similarity metrics.

Provides API key management through the console with granular rate limiting controls, allowing developers to create multiple keys with different rate limits, monitor usage, and implement quota-based access control. Rate limits are enforced per-key and per-model, enabling multi-tenant applications to allocate quotas to different users or services.

Enables models to request execution of external functions by generating structured function calls that map to a developer-provided tool registry. The implementation works by including function schemas in the system prompt, training the model to output function calls in a standardized format (name + arguments), and the API client automatically routes these calls to registered handlers. Supports parallel function execution, nested calls, and automatic result injection back into the conversation context for multi-turn reasoning.

Specialized code generation model (Codestral) fine-tuned on large code corpora to generate, complete, and explain code across 80+ programming languages. The model understands syntax, semantics, and common patterns, enabling context-aware completions that respect existing code style and architecture. Supports both fill-in-the-middle (FIM) mode for inline completions and standard left-to-right generation for new code. Integrates with IDE plugins and can be used for code review, refactoring suggestions, and test generation.

Vision-capable model (Pixtral) that processes images alongside text to answer questions, describe content, perform OCR, and analyze visual data. The implementation accepts images as base64-encoded data or URLs, processes them through a vision encoder that extracts spatial and semantic features, and fuses these representations with text embeddings for joint reasoning. Supports multiple images per request and can handle documents, screenshots, diagrams, and photographs with high accuracy.

Enables training Mistral models on custom datasets to adapt them for specific domains, writing styles, or task-specific behaviors. The fine-tuning process uses supervised learning on labeled examples (prompt-response pairs), with the API handling data validation, training orchestration, and model checkpointing. Supports both full fine-tuning and parameter-efficient methods (LoRA), with training jobs running asynchronously and results available as new model endpoints. Includes automatic data quality checks and training metrics.

Asynchronous batch API that processes multiple requests in a single job, optimizing throughput and reducing per-token costs by 50% compared to real-time API calls. Requests are queued, processed in batches during off-peak hours, and results are returned via webhook or polling. The implementation groups requests into efficient batches, reuses computational resources across similar queries, and provides detailed job status tracking and result retrieval.

Mistral infrastructure is hosted in the European Union with data residency guarantees, ensuring that all API requests, model weights, and outputs remain within EU borders. This is implemented through dedicated EU data centers, contractual commitments, and compliance with GDPR, ensuring that sensitive data never transits through or is stored in non-EU jurisdictions. Particularly valuable for regulated industries and organizations with strict data localization requirements.

API endpoint that counts tokens in text without executing inference, enabling accurate cost estimation before making API calls. The implementation uses the same tokenizer as the inference models, ensuring consistency between estimated and actual token usage. Supports batch token counting for multiple texts and provides breakdowns by message role (system, user, assistant) for multi-turn conversations.

Real-time response streaming using Server-Sent Events (SSE) protocol, allowing clients to receive model output token-by-token as it's generated rather than waiting for the complete response. The implementation maintains an open HTTP connection, sends tokens as they're generated, and includes metadata (token probabilities, finish reasons) in each event. Enables responsive UX for chat applications and allows early termination if desired output is reached before completion.