Multi Task Learning With Shared Representations And Task Specific Heads

via “multi-task learning and auxiliary objective training”

fill-mask model by undefined. 1,70,11,810 downloads.

Unique: RoBERTa's improved pretraining produces representations with stronger task-agnostic semantic content, enabling more effective multi-task learning with less task interference compared to BERT — auxiliary tasks improve primary task performance by 1-3% absolute on average

vs others: More effective for multi-task learning than single-task fine-tuning due to stronger base representations; requires more careful tuning than task-specific models but provides better generalization and inference efficiency than ensemble approaches