Panoptic Segmentation With Stuff And Thing Fusion

via “semantic segmentation with 171 extended object/stuff categories via coco-stuff variant”

330K images with object detection, segmentation, and captions.

Unique: 171-category taxonomy combining 80 instance objects + 91 stuff categories enables panoptic segmentation in single dataset; pixel-level masks for stuff enable dense scene understanding without instance boundaries

vs others: More comprehensive than ADE20K (150 categories) and larger scale than Cityscapes (5K images); unified instance+stuff annotation enables panoptic evaluation unlike separate semantic/instance datasets

via “automatic unsupervised mask generation for image panoptic segmentation”

Meta's foundation model for visual segmentation.

Unique: Uses a grid-based sampling strategy with IoU-based non-maximum suppression to deduplicate overlapping masks, avoiding redundant inference. The stability score (computed from mask prediction variance across slight input perturbations) filters unreliable masks, improving precision without manual thresholding.

vs others: More comprehensive and accurate than traditional panoptic segmentation (e.g., Mask R-CNN + semantic segmentation) because it leverages foundation model pre-training and doesn't require category-specific training, generalizing to arbitrary object types in zero-shot fashion.

OpenMMLab detection toolbox with 300+ models.

Unique: Implements panoptic segmentation by combining instance segmentation (Mask R-CNN) for things with semantic segmentation for stuff, then fusing predictions with a learned fusion module that resolves overlaps and assigns consistent instance IDs across both prediction types

vs others: More comprehensive than instance-only segmentation because it captures both countable objects and scene context; more efficient than running separate instance and semantic models because it shares backbone features; better integrated than post-hoc fusion approaches because fusion is learned end-to-end

via “instance-segmentation-with-panoptic-decoding”

image-segmentation model by undefined. 2,48,429 downloads.

Unique: Unified OneFormer architecture produces both semantic and instance outputs from a single forward pass, avoiding the need for separate instance detection heads (e.g., RPN in Mask R-CNN). Instance IDs are derived from the unified feature space rather than region proposals, enabling end-to-end differentiable instance segmentation.

vs others: More efficient than Mask R-CNN (single forward pass vs RPN + mask head) but with slightly lower instance segmentation accuracy; more unified than Mask2Former because it handles semantic, instance, and panoptic tasks with identical architecture.

via “panoptic-segmentation-stuff-things-unification”

image-segmentation model by undefined. 90,906 downloads.

Unique: Generates panoptic outputs by decoding both semantic and instance predictions from shared transformer features, then merging via a simple algorithm: stuff classes get single instance ID per class, thing classes retain instance IDs from instance decoder. This unified approach avoids separate post-processing pipelines.

vs others: Achieves 52.3 PQ on ADE20K, outperforming Mask2Former (51.9 PQ) and DeepLabV3+/Mask R-CNN ensembles (50.2 PQ) due to joint optimization of semantic and instance tasks. However, panoptic-specific models (e.g., Panoptic FPN) can achieve comparable PQ with simpler architectures if multi-task flexibility is not required.

via “panoptic segmentation interpretation with instance grouping”

image-segmentation model by undefined. 1,19,949 downloads.

Unique: Provides panoptic segmentation through mask-based queries without separate instance detection networks, enabling joint semantic and instance understanding in a single forward pass. Unlike Mask R-CNN that requires RPN + mask head, this approach uses learned mask tokens to directly predict both semantic and instance information.

vs others: Achieves panoptic segmentation 2-3x faster than Mask R-CNN (single forward pass vs RPN + mask head) and 5-10% higher PQ (panoptic quality) on ADE20K because mask-based queries naturally handle both thing and stuff classes, whereas RPN-based methods struggle with stuff classes.