Reproducible Evaluation With Fixed Question Set

57-subject benchmark, the standard metric for comparing LLMs.

Unique: Immutable, versioned dataset published on Hugging Face ensures that any builder can download and evaluate against the exact same 15,908 questions used in published research. No question generation variance, sampling randomness, or dataset drift between evaluation runs.

vs others: More reproducible than dynamically-generated benchmarks or evaluation sets that vary between researchers; enables verification of published results and fair comparison across models and time periods.