Question Answering Metrics With Span And F1 Evaluation

via “answer span extraction and evaluation metrics for reading comprehension”

95K trivia questions requiring cross-document reasoning.

Unique: Provides multiple valid answer spans per question and ground-truth span annotations within evidence documents, enabling training of span-based extractive QA models with proper handling of answer paraphrasing. The span-level annotations allow fine-grained evaluation of reading comprehension beyond simple answer matching.

vs others: More flexible than SQuAD (which has single answer spans) by allowing multiple valid spans, and more realistic than curated datasets by including noisy documents where answer spans may be paraphrased or implicit