Multi Dimensional Toxicity Scoring For Prompt Completion Pairs

via “perspective api integration for external toxicity scoring”

8-dimension trustworthiness benchmark for LLMs.

Unique: Integrates Google's Perspective API for external toxicity validation, enabling cross-checking against industry-standard toxicity detection. Provides multiple toxicity dimensions (toxicity, severe toxicity, profanity) rather than single toxicity score.

vs others: More authoritative than local classifiers because it uses Google's widely-adopted toxicity standards, though slower and rate-limited compared to local evaluation.

via “multi-dimensional toxicity scoring for prompt-completion pairs”

100K prompts for evaluating toxic text generation.

Unique: Provides 8-dimensional toxicity scoring (not binary classification) with explicit separation of severe_toxicity, threat, insult, identity_attack, profanity, sexually_explicit, and flirtation as independent dimensions, enabling nuanced analysis of different harm types rather than aggregate toxicity only. Includes source document tracking via filename and character offsets for traceability.

vs others: More granular than binary toxicity datasets (e.g., Jigsaw Toxic Comments) by decomposing toxicity into 8 independent dimensions; more practical for model evaluation than human-annotated safety benchmarks because it provides pre-scored baselines for comparison without requiring manual annotation of model outputs.