detr-resnet-50

Performs object detection by treating detection as a direct set prediction problem using a transformer encoder-decoder architecture with a ResNet-50 CNN backbone for feature extraction. The model uses bipartite matching (Hungarian algorithm) to assign predictions to ground-truth objects, eliminating the need for hand-designed components like NMS or anchor boxes. It outputs bounding boxes and class labels directly from transformer decoder outputs without post-processing.

Extracts multi-scale visual features from input images using a pretrained ResNet-50 backbone (trained on ImageNet-1k). The backbone outputs a feature map at 1/32 resolution of the input, which is then flattened and projected into the transformer embedding space. ResNet-50 uses residual connections and batch normalization to enable training of 50-layer networks, providing a proven feature extractor that balances accuracy and computational efficiency.

Implements a transformer encoder-decoder stack where the encoder processes CNN features and the decoder uses N learned object query embeddings (typically 100) to predict a fixed-size set of detections. Each query attends to the entire feature map via multi-head self-attention, enabling the model to reason about object relationships and spatial context. The decoder outputs logits for class prediction and bounding box regression for each query, treating detection as a set prediction problem rather than spatial grid-based prediction.

Trains the model using bipartite matching between predicted detections and ground-truth objects via the Hungarian algorithm, which finds the optimal one-to-one assignment minimizing total matching cost. The cost combines classification loss (cross-entropy) and bounding box regression loss (L1 + GIoU). This eliminates the need for NMS or anchor assignment heuristics, treating detection as a pure set matching problem where the model learns to predict exactly one detection per object.

Evaluates detection performance using COCO Average Precision (AP) metrics, which measure detection quality across IoU thresholds (AP@0.5:0.95 is the primary metric). The model outputs predictions in COCO format (image_id, category_id, bbox, score) which are compared against ground-truth annotations using the official COCO evaluation script. Metrics include AP (average across IoU thresholds), AP50 (IoU=0.5), AP75 (IoU=0.75), and separate metrics for small/medium/large objects.

Performs inference by running the model forward pass and post-processing raw predictions: filtering detections by confidence score threshold, converting normalized box coordinates to pixel coordinates, and optionally applying soft-NMS for overlapping detections. The model outputs logits and box deltas which are converted to class probabilities via softmax and box coordinates via inverse normalization. Post-processing is minimal compared to anchor-based methods but still includes confidence filtering and coordinate transformation.

Enables fine-tuning the pretrained model on custom object detection datasets by unfreezing the backbone and decoder weights and training with the bipartite matching loss. The model leverages ImageNet-pretrained ResNet-50 features as initialization, reducing training time and data requirements compared to training from scratch. Fine-tuning typically requires 100-1000 annotated images depending on object complexity and domain similarity to COCO.

Processes CNN features through a transformer encoder that uses positional encodings to inject spatial information into the feature maps. The model uses sine/cosine positional encodings (similar to Vision Transformer) to encode 2D spatial positions, enabling the transformer to reason about object locations without explicit spatial priors. Features are flattened and projected into the transformer embedding space, then processed through multi-head self-attention layers that attend across the entire spatial extent.