Prodigy

Prodigy uses a decorator-based recipe system (@prodigy.recipe) where Python functions define complete annotation workflows including data loading, label schema, UI configuration, and optional model predictions. Recipes are CLI-invoked with parameters (dataset name, source file, labels) that override function defaults, enabling rapid iteration without code changes. This approach treats annotation pipelines as first-class Python objects rather than configuration files, allowing full programmatic control over data flow and task generation.

Prodigy integrates external model predictions (from spaCy, transformers, or custom models) into the annotation UI to pre-populate labels and prioritize uncertain examples. The system accepts model predictions as JSON objects in the annotation stream and uses them to score task difficulty or confidence, though the specific uncertainty sampling algorithm and model retraining loop are not publicly documented. This reduces labeling effort by surfacing high-uncertainty examples first and providing model suggestions that annotators accept/reject.

Prodigy provides a stats command (prodigy stats) that computes aggregate statistics over annotations in a dataset, including label distribution, annotation counts, and optionally agreement metrics if multiple annotators are present. The stats functionality is accessible via CLI and Python API, enabling users to monitor annotation progress and data quality without manual analysis. Statistics are computed directly from the SQLite database and can be filtered by dataset, label, or time range.

Prodigy allows users to create custom annotation interfaces by providing HTML and JavaScript that hooks into Prodigy's frontend API. Custom interfaces receive task data as JSON, render custom UI elements, and submit annotations back to Prodigy via JavaScript function calls. This enables domain-specific annotation UIs (e.g., custom graph visualization, timeline annotation, specialized medical imaging tools) without modifying Prodigy's core code. The custom interface mechanism is recipe-based and integrates with the same task streaming and database persistence as built-in interfaces.

Prodigy is tightly integrated with spaCy (same vendor, Explosion AI) and can use spaCy models to pre-populate NER annotations, provide entity suggestions, and score prediction confidence. Recipes can load spaCy models and pass predictions to the annotation UI, where annotators accept, reject, or correct suggestions. This integration is documented through case studies and examples but the specific API for spaCy model integration is not fully detailed in provided documentation.

Prodigy supports computer vision annotation tasks including drawing bounding boxes on images, creating segmentation masks, and classifying images or regions. The image annotation interface allows users to draw rectangles or polygons on images and assign labels to regions or entire images. Annotations are stored with pixel coordinates and label information, enabling export for object detection or segmentation model training. The image annotation capability is built-in but details on supported image formats, coordinate systems, and export formats are not fully documented.

Prodigy documentation mentions support for audio and video annotation as a task type, though specific details on the annotation interface, supported formats, and capabilities are not provided in available documentation. The audio/video annotation feature is listed in the docs navigation but implementation details are absent, suggesting it may be a documented but underdeveloped feature or require custom interface implementation.

Prodigy uses a lifetime license model where users pay once for perpetual access, rather than a subscription-based SaaS model. The pricing structure offers flexible options for individuals and teams, though specific pricing tiers and team size limits are not documented in available materials. This contrasts with SaaS annotation platforms that charge recurring subscription fees, making Prodigy cost-effective for long-term projects.

Prodigy provides built-in annotation interfaces for common NLP tasks (NER span labeling, text classification, relation extraction, dependencies) and computer vision tasks (image bounding boxes, segmentation). Each interface is a pre-built HTML/JavaScript component that renders annotation tasks and captures user interactions (clicks, drags, selections). Users can select which interface to use via recipe parameters, and custom interfaces can be built by providing HTML/JavaScript that hooks into Prodigy's JavaScript API for task submission and UI state management.

Prodigy stores all annotations in a SQLite database by default, with each annotation record containing the task data, user input, timestamp, and metadata. The database schema is managed automatically by Prodigy; users interact with annotations through the Python API or CLI commands (prodigy db-out, prodigy stats). The architecture supports pluggable database backends (mechanism undocumented), suggesting alternative storage systems can be integrated, though only SQLite is documented as officially supported.

Prodigy provides CLI commands (prodigy db-out) and Python API methods to export annotations from the SQLite database in formats suitable for model training. Exported data includes the original text/image, user-provided labels, and optionally model predictions and metadata. The export mechanism supports filtering by dataset, label, or other criteria, enabling users to prepare training datasets without manual data wrangling. Specific export formats (JSONL, spaCy training format, etc.) are mentioned in case studies but not fully documented.

Prodigy supports review and comparison tasks where annotators evaluate existing annotations or compare model outputs side-by-side. The review interface allows users to accept, reject, or correct previous annotations, and A/B evaluation tasks can present two model predictions or annotation variants for comparison. This capability is built into the task routing system, enabling conditional workflows where review tasks are triggered based on annotation disagreement, model confidence, or other criteria.

Prodigy runs entirely on the user's machine or self-hosted infrastructure with no required connectivity to Prodigy's servers. The web UI is served locally (typically localhost:8080), and all data (annotations, models, configurations) remain on the user's hardware. Prodigy explicitly does not collect telemetry or phone home, making it suitable for privacy-sensitive or air-gapped environments. This architecture contrasts with SaaS platforms that require cloud accounts and data transmission.

Prodigy supports task routing where annotation workflows branch based on conditions (e.g., if model confidence is low, route to review task; if entity type is rare, route to specialized annotator). The routing mechanism is recipe-based and uses Python logic to determine which task to present next based on previous annotations or model predictions. This enables complex workflows where different examples follow different annotation paths without requiring separate datasets or manual task assignment.

Prodigy exposes a Python API (in addition to CLI) for creating datasets, adding annotations, querying the database, and managing annotation workflows programmatically. Users can import Prodigy as a Python library and call functions to interact with the database, enabling integration with Jupyter notebooks, training scripts, and custom automation. This API-first design allows annotation workflows to be embedded in larger ML pipelines without shell scripting or manual data export.

Prodigy recipes can define data loaders that stream annotation tasks from various sources (JSONL files, CSV, image folders, APIs, databases) without loading the entire dataset into memory. Tasks are generated on-demand as the annotator progresses through the UI, enabling annotation of datasets larger than available RAM. The streaming architecture also supports integration with model prediction APIs, where predictions are fetched on-demand for each task rather than pre-computed.