Cross-Domain Few-Shot Object Detection via Enhanced Open-Set Object Detector

TL;DR

This paper employs measures including style, inter-class variance (ICV), and indefinable boundaries (IB) to understand the domain gap, and proposes several novel modules to address issues, significantly improving upon the base DE-ViT.

Abstract

This paper studies the challenging cross-domain few-shot object detection (CD-FSOD), aiming to develop an accurate object detector for novel domains with minimal labeled examples. While transformer-based open-set detectors, such as DE-ViT, show promise in traditional few-shot object detection, their generalization to CD-FSOD remains unclear: 1) can such open-set detection methods easily generalize to CD-FSOD? 2) If not, how can models be enhanced when facing huge domain gaps? To answer the first question, we employ measures including style, inter-class variance (ICV), and indefinable boundaries (IB) to understand the domain gap. Based on these measures, we establish a new benchmark named CD-FSOD to evaluate object detection methods, revealing that most of the current approaches fail to generalize across domains. Technically, we observe that the performance decline is associated with our proposed measures: style, ICV, and IB. Consequently, we propose several novel modules to address these issues. First, the learnable instance features align initial fixed instances with target categories, enhancing feature distinctiveness. Second, the instance reweighting module assigns higher importance to high-quality instances with slight IB. Third, the domain prompter encourages features resilient to different styles by synthesizing imaginary domains without altering semantic contents. These techniques collectively contribute to the development of the Cross-Domain Vision Transformer for CD-FSOD (CD-ViTO), significantly improving upon the base DE-ViT. Experimental results validate the efficacy of our model.

Figures (6)

• Figure 1: (a) Our motivation: The DE-ViT open-set detector excels in FSOD but struggles in CD-FSOD, inspiring our creation of CD-ViTO. (b) Technical motivation: FSOD models face challenges when dealing with cross-domain targets, such as small inter-class variance (ICV), indefinable boundaries (IB), and varying appearances (styles).
• Figure 2: CD-FSOD benchmark: COCO serves as the training source data, while six datasets are utilized as novel testing target datasets. These datasets exhibit variations in styles, inter-class variance (ICV), and indefinable boundaries (IB).
• Figure 3: (a) Overall framework of our CD-ViTO. We build our method upon an open-set detector (DE-ViT). Modules in blue are inherited from DE-ViT while modules in orange are proposed by us. New improvements include learnable instance features, instance reweighting, domain prompter, and finetuning; (b) Illustration of our modules.
• Figure 4: Ablation study on our proposed modules.
• Figure 5: Visualization results of DE-ViT, our CD-ViTO, and ground truth.