Adversarially Trained Object Detector for Unsupervised Domain Adaptation

TL;DR

This study establishes that adversarially trained detectors achieve improved detection performance in target domains that are significantly shifted from source domains, and proposes a method that combines adversarial training and feature alignment to ensure the improved alignment of robust features with the target domain.

Abstract

Unsupervised domain adaptation, which involves transferring knowledge from a label-rich source domain to an unlabeled target domain, can be used to substantially reduce annotation costs in the field of object detection. In this study, we demonstrate that adversarial training in the source domain can be employed as a new approach for unsupervised domain adaptation. Specifically, we establish that adversarially trained detectors achieve improved detection performance in target domains that are significantly shifted from source domains. This phenomenon is attributed to the fact that adversarially trained detectors can be used to extract robust features that are in alignment with human perception and worth transferring across domains while discarding domain-specific non-robust features. In addition, we propose a method that combines adversarial training and feature alignment to ensure the improved alignment of robust features with the target domain. We conduct experiments on four benchmark datasets and confirm the effectiveness of our proposed approach on large domain shifts from real to artistic images. Compared to the baseline models, the adversarially trained detectors improve the mean average precision by up to 7.7%, and further by up to 11.8% when feature alignments are incorporated. Although our method degrades performance for small domain shifts, quantification of the domain shift based on the Frechet distance allows us to determine whether adversarial training should be conducted.

Figures (6)

• Figure 1: Visualization of robust and non-robust features. The standard trained model in the source domain is highly dependent on non-robust features, which are not informative for the largely shifted target domain. In contrast, the robust features acquired by the adversarially trained model are informative even in the largely shifted target domain.
• Figure 2: Framework of the proposed method. $F_1$ and $F_2$ are object detection networks, $D$ is a domain discriminator, and GRL is a gradient reversal layer. First, we propagate the source images with the initial perturbations $\bm{\delta}_{0}$ and then compute the adversarial perturbations $\bm{\delta^*}$ using the gradients of the losses. Then, adversarial training on the source images perturbed by $\bm{\delta^*}$ and adversarial feature learning on the source and target images are performed.
• Figure 3: Examples of the datasets used in the experiments.
• Figure 4: Examples of style transfer via AdaIN for the PASCAL VOC test set. The test set is stylized into three style images by varying the content--style trade-off $\beta$.
• Figure 5: Fréchet distance (FD) between the PASCAL VOC training set and the stylized PASCAL VOC test set, and the ratio of the mAP of AT to mAP of ST on the stylized test set. The larger the FD, the better the detection performance of AT on the stylized test sets.