Predictive Performance Scoring

via “efficient-multi-prompt-evaluation-with-performance-prediction”

PromptBench is a powerful tool designed to scrutinize and analyze the interaction of large language models with various prompts. It provides a convenient infrastructure to simulate **black-box** adversarial **prompt attacks** on the models and evaluate their performances.

Unique: Uses a sample-based prediction approach where a small subset of prompt-model-output pairs trains a lightweight predictor to estimate full-dataset performance, rather than evaluating all prompts. This enables order-of-magnitude speedups for multi-prompt evaluation while maintaining reasonable accuracy.