LLM Guard

Implements a modular scanner framework where both input (pre-LLM) and output (post-LLM) validators follow a common interface returning (sanitized_text, is_valid, risk_score) tuples. Scanners are composed independently and can be chained in arbitrary order, enabling flexible security pipelines. The architecture decouples scanner logic from orchestration, allowing developers to enable/disable scanners via configuration without code changes.

Detects prompt injection attacks using a multi-strategy approach combining regex-based pattern matching for known injection signatures, semantic similarity analysis against injection templates, and structural analysis of prompt delimiters and role-switching patterns. The scanner identifies attempts to override system instructions, inject new directives, or manipulate LLM behavior through adversarial prompt crafting.

Supports ONNX (Open Neural Network Exchange) optimization for transformer-based scanners, enabling faster inference and reduced memory footprint. Converts HuggingFace models to ONNX format with quantization options (int8, float16), enabling deployment on CPU-only or edge devices. Configuration-driven ONNX enablement allows switching between full-precision and optimized models without code changes. Reduces model inference latency by 2-10x compared to PyTorch, enabling real-time scanning in latency-sensitive applications.

Enables developers to compose scanners into custom security pipelines via configuration files (YAML) or code, selecting which scanners to enable, their order, and their parameters. Supports conditional scanner execution (e.g., run PII scanner only if prompt contains certain keywords), scanner chaining (output of one scanner feeds into next), and policy-driven behavior (different scanner sets for different user roles or risk profiles). Eliminates need to write custom orchestration code for complex security workflows.

Provides hooks for logging and monitoring all scanning decisions, enabling compliance auditing and security analysis. Integrates with standard Python logging framework and supports custom observability backends. Logs include scanner name, input text, risk score, sanitization actions, and decision (allow/block). Enables teams to audit security decisions, identify patterns in attacks, and monitor scanner performance. Supports structured logging (JSON) for integration with log aggregation systems (ELK, Datadog, Splunk).

Supports scanning multiple prompts or outputs in a single API call, enabling efficient batch processing for high-throughput scenarios. Processes batches through the scanner pipeline with optimized tensor operations and optional parallelization, reducing per-item overhead compared to individual requests.

Aggregates risk scores from multiple scanners using configurable strategies (weighted sum, maximum, AND/OR logic) to produce a final security decision. Enables policy-based rules (e.g., 'block if any scanner scores > 0.8 OR toxicity > 0.9') for nuanced security decisions beyond binary allow/block.

Detects personally identifiable information (names, emails, phone numbers, SSNs, credit cards, etc.) in prompts and outputs using pattern matching and NER (Named Entity Recognition) models. Detected PII can be anonymized by replacing with tokens and storing original values in a stateful Vault object, enabling later de-anonymization. The Vault class maintains in-memory or persistent storage of PII mappings, supporting workflows where sensitive data must be redacted from LLM context but recovered in responses.

Detects toxic, abusive, and harmful language in prompts and outputs using transformer-based text classification models trained on toxicity datasets. Scanners classify text into categories (profanity, insults, threats, harassment) and assign risk scores. Developers can configure severity thresholds to reject or flag content based on risk tolerance, enabling fine-grained control over what language is permitted in different contexts.

Detects and filters prompts/outputs containing banned topics or sensitive subjects (e.g., violence, self-harm, illegal activities, adult content) based on configurable policy lists. Uses semantic similarity matching against topic keywords and phrases to identify content violating organizational policies. Developers can define custom banned topic lists per deployment, enabling different policies for different user segments or jurisdictions.

Detects attempts to inject executable code (SQL, shell commands, Python, JavaScript) into prompts or malicious code in LLM outputs. Uses pattern matching for common injection signatures (SQL keywords, shell metacharacters), AST parsing for code structure analysis, and optional semantic analysis to identify code-like patterns. Prevents LLM from being used to generate or execute malicious code, and blocks prompts attempting to manipulate backend systems through code injection.

Detects attempts to hide malicious content using invisible unicode characters, zero-width characters, homoglyph attacks, and other encoding-based obfuscation techniques. Analyzes character encodings to identify non-printable characters, combining marks, and lookalike characters that could bypass other security scanners or confuse users. Prevents attackers from using unicode tricks to inject prompts or hide malicious instructions in LLM outputs.

Validates that prompts and outputs fit within LLM context windows by tokenizing text using language-specific tokenizers (HuggingFace, OpenAI, Anthropic). Calculates token counts for prompts and outputs, enforces maximum token limits, and provides warnings when approaching context window limits. Integrates with multiple tokenizer backends, enabling accurate token counting for different LLM providers without sending data to external APIs.

Exposes LLM Guard scanners via FastAPI HTTP endpoints, enabling remote deployment of security scanning as a microservice. The API service (llm-guard-api) wraps scanner implementations with REST endpoints for prompt validation, output validation, and batch scanning. Supports configuration-driven scanner selection via YAML, Docker deployment with GPU acceleration, and observability hooks for logging and monitoring. Enables teams to deploy scanning infrastructure separately from application code.

Provides native integration with LiteLLM proxy, enabling automatic injection of LLM Guard scanners into LLM API calls without modifying application code. Scanners run transparently before prompts reach the LLM and after responses are generated, implementing the dual-gate security model. Configuration-driven scanner selection allows different scanning policies per model, provider, or user without code changes. Supports all LiteLLM-compatible providers (OpenAI, Anthropic, Ollama, etc.).