Ambiguity Aware Mask Generation With Multiple Candidate Outputs

via “masked language model token prediction via bidirectional transformer attention”

fill-mask model by undefined. 11,20,072 downloads.

Unique: Implements true bidirectional context modeling through masked language modeling pretraining (unlike GPT's unidirectional approach), using WordPiece subword tokenization with 30,522 tokens and 24-layer transformer with 16 attention heads, trained on BookCorpus + Wikipedia for 1M steps with dynamic masking strategy

vs others: Outperforms RoBERTa and ELECTRA on GLUE benchmarks for token prediction tasks due to larger pretraining corpus, but slower inference than DistilBERT (40% parameter reduction) and less multilingual coverage than mBERT

via “multi-prompt mask disambiguation and refinement”

Python AI package: segment-anything

Unique: Integrates IoU prediction heads into the mask decoder, allowing the model to estimate mask quality without ground truth — enabling confidence-based ranking and automatic selection of best masks, a capability absent in standard segmentation models that only output masks without quality estimates

vs others: Provides built-in confidence scoring for masks (IoU predictions) whereas traditional segmentation models require external validation; enables interactive refinement without retraining, unlike active learning approaches that require model updates

via “ambiguity-aware mask generation with multiple candidate outputs”

* ⭐ 04/2023: [DINOv2: Learning Robust Visual Features without Supervision (DINOv2)](https://arxiv.org/abs/2304.07193)

Unique: Explicitly models segmentation ambiguity by training the decoder to produce multiple valid masks with confidence scores, rather than forcing a single deterministic output. This design acknowledges that some prompts are inherently ambiguous and provides mechanisms for downstream systems to handle uncertainty without resorting to post-hoc ensemble methods.

vs others: More principled than post-hoc ensemble methods because ambiguity is modeled during training, enabling the decoder to learn which prompts are inherently ambiguous and generate appropriate candidate sets, while confidence scores provide calibrated uncertainty estimates.