CDFormer: Cross-Domain Few-Shot Object Detection Transformer Against Feature Confusion

TL;DR

CDFormer tackles cross-domain few-shot object detection by addressing feature confusion with two modules: Object-Background Distinguishing (OBD) and Object-Object Distinguishing (OOD). OBD uses a learnable background token $\mathbf{T}_{\text{b}}$ to separate background from objects, with Object Feature Enhancement (OFE) and Background Feature Learning (BFL) to refine representations. OOD introduces a class embedding space $\mathbf{T}=\{\mathbf{t}_1,\dots,\mathbf{t}_C\}$ and applies the InfoNCE loss $\mathcal{L}_{\text{OOD}}= -\frac{1}{C} \sum_{i=1}^{C} \log \frac{\exp(\mathbf{f}_i^\top \mathbf{t}_i / \tau)}{\sum_{j=1}^C \exp(\mathbf{f}_i^\top \mathbf{t}_j / \tau)}$ to align support features $\mathbf{f}_i$ with their corresponding embeddings while pushing others apart, with temperature $\tau$. The approach builds on a DETR-style transformer, redefines the detection head for class-agnostic meta-learning, and uses background placeholders to handle unknown numbers of target classes. Experiments pretrained on COCO and evaluated on six CD-FSOD datasets show state-of-the-art results, with large gains on challenging domains such as NEU-DET and UODD, including improvements of +12.9, +11.0, and +10.4 mAP points at 1/5/10 shots under fine-tuning.

Abstract

Cross-domain few-shot object detection (CD-FSOD) aims to detect novel objects across different domains with limited class instances. Feature confusion, including object-background confusion and object-object confusion, presents significant challenges in both cross-domain and few-shot settings. In this work, we introduce CDFormer, a cross-domain few-shot object detection transformer against feature confusion, to address these challenges. The method specifically tackles feature confusion through two key modules: object-background distinguishing (OBD) and object-object distinguishing (OOD). The OBD module leverages a learnable background token to differentiate between objects and background, while the OOD module enhances the distinction between objects of different classes. Experimental results demonstrate that CDFormer outperforms previous state-of-the-art approaches, achieving 12.9% mAP, 11.0% mAP, and 10.4% mAP improvements under the 1/5/10 shot settings, respectively, when fine-tuned.

Figures (7)

• Figure 1: Illustration of the task, challenges, and our performance of the CD-FSOD. (a) Object-background confusion. (b) Object-object confusion. (c) Performance comparison between our CDFormer and the state-of-the-art DE-ViT DE-ViT and CD-ViTO CD-ViTO.
• Figure 2: The overall architecture of our CDFormer. The core of CDFormer consists of the object-background distinguishing (OBD) module and the object-object distinguishing (OOD) module. The term target classes refers to the object categories from the dataset represented in the support set.
• Figure 3: Top 5 results of the COCO-pretrained RPN on same-domain and cross-domain datasets. Red boxes indicate RPN predictions, and blue boxes indicate ground truth. (a) RPN performs well on the same-domain VOC dataset. (b) RPN exhibits significant performance degradation on cross-domain datasets.
• Figure 4: Object feature enhancement (OFE) unit. The OFE serves different purposes in the query image encoder branch and the support set encoder branch.
• Figure 5: Background feature learning (BFL) unit. As an example, the figure depicts a support set with four target classes and one background placeholder.