7 Category Safety Taxonomy With Fine Grained Failure Mode Classification

via “seven-category safety taxonomy and question curation”

11K safety evaluation questions across 7 categories.

Unique: Explicitly defines 7 non-overlapping safety categories and curates 11,435 questions to cover them systematically, providing a structured taxonomy rather than ad-hoc safety testing. The taxonomy is comprehensive enough to cover major harm types (physical, mental, legal, ethical, privacy) while remaining tractable for evaluation.

vs others: More comprehensive and structured than single-category benchmarks (e.g., toxicity-only); provides a holistic safety assessment framework that aligns with regulatory and safety research perspectives.

via “7-category safety taxonomy with fine-grained failure mode classification”

11K safety evaluation questions across 7 categories.

Unique: Implements 7-category safety taxonomy with category-balanced few-shot examples enabling systematic failure mode diagnosis. Most safety benchmarks (TruthfulQA, HarmBench) report only aggregate safety scores without category-level breakdown or category-specific few-shot examples.

vs others: Category stratification reveals which safety domains models struggle with, enabling targeted improvements; category-balanced few-shot examples support category-specific evaluation unlike benchmarks with random few-shot sampling.

via “taxonomy-based unsafe content categorization”

Llama Guard 4 is a Llama 4 Scout-derived multimodal pretrained model, fine-tuned for content safety classification. Similar to previous versions, it can be used to classify content in both LLM...

Unique: Uses instruction-tuned fine-tuning on safety-labeled data to produce multi-dimensional category scores in a single forward pass, rather than training separate binary classifiers per category or using rule-based heuristics. Inherits Llama Guard 3's taxonomy design but extends it with visual understanding.

vs others: Provides granular per-category scores in one API call, enabling policy-based routing, whereas binary classifiers (safe/unsafe) require downstream logic to determine which violation type occurred, and rule-based systems are brittle to paraphrasing.