Adversarial Robustness Testing

via “red teaming and adversarial test case generation”

LLM evaluation framework — 14+ metrics, faithfulness/hallucination detection, Pytest integration.

Unique: Implements red teaming as a specialized evaluation mode that uses LLM-as-judge to generate adversarial inputs following specific attack patterns (prompt injection, jailbreak, bias probing), then evaluates system responses using safety metrics; integrates with the standard evaluation pipeline for tracking and reporting

vs others: More systematic than manual red teaming because it uses LLM-guided generation to explore adversarial input space and automatically evaluates responses against safety metrics, enabling scalable adversarial testing