Deformable Cross Attention Fusion

via “cross-attention fusion of image features and prompt embeddings”

Meta's foundation model for visual segmentation.

Unique: Uses bidirectional cross-attention where both prompts attend to image features and image features attend to prompts, enabling mutual refinement. This design allows prompts to disambiguate image regions and image context to refine prompt interpretation.

vs others: More principled than concatenation-based fusion because attention learns which image regions are relevant to each prompt, avoiding feature dilution from irrelevant image regions and enabling explicit multi-prompt composition.

via “masked attention-based segmentation head with deformable cross-attention”

image-segmentation model by undefined. 1,55,904 downloads.

Unique: Replaces dense convolution-based decoders with learnable class queries that use deformable cross-attention to dynamically sample relevant spatial locations, reducing computation from O(HW) to O(HW·k) where k is number of deformable sampling points — fundamentally different from FCN/DeepLab's dense prediction approach

vs others: Achieves better accuracy-latency tradeoff than dense decoders (82.0 mIoU at 250ms vs DeepLabV3+ at 79.6 mIoU at 180ms) through learned spatial focus, though adds complexity in query initialization and training stability

via “deformable-cross-attention-fusion”

image-segmentation model by undefined. 90,906 downloads.

Unique: Extends deformable convolution principles to cross-attention by learning per-query offset predictions that sample from reference feature maps at adaptive 2D coordinates. Unlike fixed grid sampling, each query position learns which spatial regions to attend to, enabling content-aware feature fusion without explicit multi-head processing.

vs others: Reduces attention computation by 30-40% vs standard multi-head cross-attention while improving boundary precision by 1-2 mIoU on ADE20K, as learned offsets naturally align with object edges and fine structures that fixed attention patterns would miss.

via “multi-scale-decoder-with-cross-attention-fusion”

image-segmentation model by undefined. 54,407 downloads.

Unique: Uses learnable query embeddings with multi-head cross-attention to progressively fuse features from all 4 backbone scales, with separate attention heads specializing in different scales. Unlike FPN-based decoders that use fixed upsampling, this approach learns adaptive feature weighting that varies spatially and by task.

vs others: Achieves 3-5% higher mIoU on small objects compared to FPN-based decoders because attention mechanisms can dynamically emphasize high-resolution features where needed, while maintaining competitive performance on large objects.

via “transformer-based context aggregation across spatial regions”

object-detection model by undefined. 1,06,918 downloads.

Unique: Deformable transformer attention adaptively samples spatial regions based on learned offsets, enabling efficient long-range context aggregation without quadratic complexity of standard attention. This is architecturally distinct from dense transformer detectors (DETR) that attend to all spatial locations uniformly.

vs others: Captures long-range spatial relationships better than CNN-based detectors (YOLO, Faster R-CNN) with limited receptive fields, while remaining more efficient than vanilla transformers (DETR) through deformable sampling that reduces attention complexity from O(HW)² to O(HW·k) where k is small sample count.