Mask Based Iterative Segmentation With Hint Propagation

via “iterative mask refinement with cross-attention prompt fusion”

Meta's foundation model for visual segmentation.

Unique: Implements iterative mask refinement through cross-attention between prompt embeddings and image features, enabling the decoder to dynamically adjust focus based on user feedback without retraining. The architecture supports mixed prompt types through unified embedding spaces, allowing points, boxes, and masks to be processed jointly in a single attention computation.

vs others: More efficient than retraining models for each user correction (as in active learning approaches), and more intuitive than parameter adjustment because users provide direct spatial feedback rather than tuning hyperparameters.