Presidio

Detects 30+ PII entity types (names, SSNs, credit cards, phone numbers, bitcoin wallets, etc.) in unstructured text using a pluggable recognizer system that combines NLP-based entity extraction, regex pattern matching, and machine learning models. The Analyzer component orchestrates multiple recognizers in sequence, applies context enhancement to reduce false positives, and returns scored entity matches with confidence levels and character offsets for precise redaction.

Provides an extensible architecture for building custom PII recognizers by implementing a base Recognizer interface and registering them with the Analyzer. Developers can create domain-specific recognizers using regex patterns, spaCy NLP pipelines, external ML models, or API calls (e.g., calling a custom ML service to detect proprietary entity types). The framework handles recognizer composition, scoring aggregation, and context passing without requiring framework modifications.

Defines a standardized entity type taxonomy (PERSON, EMAIL, PHONE_NUMBER, CREDIT_CARD, SSN, LOCATION, ORGANIZATION, etc.) that is language-agnostic and extensible. Built-in recognizers target these entity types, and custom recognizers can define new types (e.g., EMPLOYEE_ID, MEDICAL_RECORD_NUMBER). Entity types are used for operator mapping (e.g., 'PERSON -> redact'), confidence thresholding, and filtering. The system supports entity type hierarchies (e.g., PERSON is a subtype of IDENTITY).

Provides pre-built Docker images for Analyzer, Anonymizer, and Image Redactor components that can be deployed as microservices. Includes Docker Compose configurations for local development and Kubernetes manifests for production deployments. Supports scaling individual components independently, health checks, and integration with container orchestration platforms. Enables rapid deployment without manual Python environment setup.

Supports PII detection across multiple languages (English, Spanish, Portuguese, French, German, Chinese, Dutch, Greek, Italian, Lithuanian, Norwegian, Polish, Romanian, Russian, Ukrainian) through pluggable spaCy language models. Allows users to specify language per analysis or auto-detect language. Supports custom NLP models by implementing a custom NLP engine interface. Enables language-specific context enhancement and recognizer rules.

De-identifies detected PII entities using a pluggable operator framework that supports multiple anonymization strategies: replace (with fixed/random values), redact (mask with asterisks), hash (deterministic hashing for consistency), encrypt (reversible encryption with key management), mask (partial masking like XXX-XX-1234), and custom operators. The Anonymizer component applies operators to text based on entity type mappings, preserves non-PII content, and supports deanonymization for authorized users via encrypted operator state.

Detects and redacts PII in image files (PNG, JPG) and medical DICOM images by extracting text via Optical Character Recognition (OCR), running the extracted text through the Analyzer to identify PII entities, and then redacting those regions in the original image using bounding boxes. The Image Redactor component handles image format conversion, OCR engine integration (Tesseract or cloud-based), and supports both text-based and visual redaction (blurring, pixelation) for DICOM images with medical-specific entity types.

Detects and anonymizes PII in structured datasets (CSV, JSON, Parquet, databases) by applying the Analyzer to column values, mapping detected entities to anonymization operators, and writing de-identified output in the same format. The Structured component handles schema inference, batch processing of large files, and supports both column-level (redact entire column) and cell-level (redact specific values) anonymization strategies. Integrates with PySpark for distributed processing of multi-gigabyte datasets.

Exposes Presidio's core components (Analyzer, Anonymizer, Image Redactor) as RESTful microservices via Flask/FastAPI, enabling integration into larger systems without Python dependencies. Each component runs in a separate Docker container (ports 5002 for Analyzer, 5001 for Anonymizer, 5003 for Image Redactor) with independent scaling, and supports Kubernetes deployment with auto-scaling, health checks, and service discovery. The REST API abstracts implementation details and enables polyglot integration (Java, Go, Node.js, etc.).

Assigns confidence scores (0-1) to detected PII entities based on recognizer agreement, context analysis, and entity-type-specific patterns. The Analyzer aggregates scores from multiple recognizers (NLP, regex, custom) and applies context enhancement to reduce false positives (e.g., 'John' in 'John Smith' is more likely a name than 'John' as a standalone word). Supports per-entity-type confidence thresholds, enabling fine-grained control (e.g., require 0.9 confidence for SSNs but accept 0.5 for names).

Enables authorized users to reverse anonymization applied by encrypt operators by storing encrypted operator state (encryption keys, salt values) alongside anonymized data. The Deanonymizer component uses stored state to decrypt PII values, supporting integration with external key management systems (Azure Key Vault, AWS KMS, HashiCorp Vault) for secure key storage and rotation. Supports audit logging of deanonymization requests for compliance.

Allows non-developers to configure Presidio's behavior via YAML files without writing Python code. YAML configuration specifies which recognizers to enable, entity-type-to-operator mappings (e.g., 'PERSON -> redact', 'SSN -> encrypt'), confidence thresholds, and custom entity types. The framework loads YAML at startup and applies configurations without code changes, enabling rapid experimentation and deployment of policy changes.

Processes large text/image/structured data files in batches with configurable batch size, progress tracking, and graceful error handling. The framework processes each batch independently, reports progress (items processed, items failed, estimated time remaining), and continues processing on errors (e.g., skips malformed images, logs errors, continues with next batch). Supports parallel batch processing via multiprocessing or PySpark for distributed execution.