CAT: Exploiting Inter-Class Dynamics for Domain Adaptive Object Detection

TL;DR

This work tackles domain adaptive object detection under severe class imbalance by introducing Class-Aware Teacher (CAT). CAT learns inter-class biases with an Inter-Class Relation module (ICRm), augments minority representation via Class-Relation Augmentation (CRA) using MixUp on related class crops stored in a Cropbank, and applies an Inter-Class Loss (ICL) to emphasize hard minority cases. The approach yields state-of-the-art results on Cityscapes to Foggy Cityscapes (52.5 mAP, up from 51.2) and strong gains on PASCAL VOC to Clipart1K, with ablations confirming the efficacy of each component. By explicitly modeling inter-class dynamics and cross-domain augmentation, CAT provides a principled framework to mitigate minority class bias in DAOD with practical impact for real-world domain shifts.

Abstract

Domain adaptive object detection aims to adapt detection models to domains where annotated data is unavailable. Existing methods have been proposed to address the domain gap using the semi-supervised student-teacher framework. However, a fundamental issue arises from the class imbalance in the labelled training set, which can result in inaccurate pseudo-labels. The relationship between classes, especially where one class is a majority and the other minority, has a large impact on class bias. We propose Class-Aware Teacher (CAT) to address the class bias issue in the domain adaptation setting. In our work, we approximate the class relationships with our Inter-Class Relation module (ICRm) and exploit it to reduce the bias within the model. In this way, we are able to apply augmentations to highly related classes, both inter- and intra-domain, to boost the performance of minority classes while having minimal impact on majority classes. We further reduce the bias by implementing a class-relation weight to our classification loss. Experiments conducted on various datasets and ablation studies show that our method is able to address the class bias in the domain adaptation setting. On the Cityscapes to Foggy Cityscapes dataset, we attained a 52.5 mAP, a substantial improvement over the 51.2 mAP achieved by the state-of-the-art method.

Figures (6)

• Figure 1: Performance of Class-Aware Teacher (CAT). AT at (top left), with Inter-Class Loss, ICL, (top-right), with Class Relation Augmentation, CRA, (bottom-left), and CAT (bottom-right). CAT is able to address misclassification and false positives, blue and red boxes, respectively, in minority classes such as 'train'. The combination of ICL and CRA further boosts performance by reducing the number of false positives shown as pink boxes.
• Figure 2: (a) Class-Aware Teacher (CAT) consists of: a student-teacher network; Inter-Class Relation module (ICRm), which estimates inter-class biases; Class-Relation Augmentation, which augments images to reduce the inter-class biases by mixing the cropped instances of related classes; and Inter-Class Loss, which emphasises the loss on highly misclassified minority classes. (b) Class-Relation Augmentation demonstrated on majority (Car) and minority (Bus) classes.
• Figure 3: Qualitative results of CAT. We show the results of AT and CAT on the top and bottom, respectively. CAT is able to address misclassification (col 1,2,4), false negatives (col 1,3), and false positives (col 1,3,4). Box colour represents: Green $\to$ true positives, Blue $\to$ misclassified, Red $\to$ false negatives, Pink $\to$ false positives.
• Figure 4: Class distribution of datasets used for the Cityscapes $\to$ Foggy Cityscapes task. We can see that person and car classes form the majority of all classes. The distribution of classes for the labeled dataset and validation set is similar which makes for an simpler task.
• Figure 5: Class distribution of datasets used for the PASCAL VOC $\to$ Clipart1k task. Person is a majority class for all datasets, however other classes for PASCAL VOC have a similar number of instance. The imbalance is stronger in the Clipart1K dataset with classes such as motorbike and bus being a minority.