DualTeacher: Bridging Coexistence of Unlabelled Classes for Semi-supervised Incremental Object Detection

TL;DR

The paper tackles semi-supervised incremental object detection (SSIOD), where new classes arrive with limited labels amid large amounts of unlabelled data containing both old and new classes. It identifies the failure of standard knowledge distillation in SSIOD due to conflicting predictions from coexisting unlabelled instances and proposes DualTeacher, which deploys two specialized teachers for old and new classes and uses the concatenation of their predictions as pseudo-labels to train the student, updating the new teacher via EMA without adding hyperparameters. Evaluations on MS-COCO-based SSIOD benchmarks show substantial gains over strong IOD baselines, with improvements up to $18.28$ AP and better retention of old-class performance, even with limited supervision. The approach is practical, plug-in, and scalable across task splits and labeling ratios, offering a new baseline and direction for SSIOD in real-world settings.

Abstract

In real-world applications, an object detector often encounters object instances from new classes and needs to accommodate them effectively. Previous work formulated this critical problem as incremental object detection (IOD), which assumes the object instances of new classes to be fully annotated in incremental data. However, as supervisory signals are usually rare and expensive, the supervised IOD may not be practical for implementation. In this work, we consider a more realistic setting named semi-supervised IOD (SSIOD), where the object detector needs to learn new classes incrementally from a few labelled data and massive unlabelled data without catastrophic forgetting of old classes. A commonly-used strategy for supervised IOD is to encourage the current model (as a student) to mimic the behavior of the old model (as a teacher), but it generally fails in SSIOD because a dominant number of object instances from old and new classes are coexisting and unlabelled, with the teacher only recognizing a fraction of them. Observing that learning only the classes of interest tends to preclude detection of other classes, we propose to bridge the coexistence of unlabelled classes by constructing two teacher models respectively for old and new classes, and using the concatenation of their predictions to instruct the student. This approach is referred to as DualTeacher, which can serve as a strong baseline for SSIOD with limited resource overhead and no extra hyperparameters. We build various benchmarks for SSIOD and perform extensive experiments to demonstrate the superiority of our approach (e.g., the performance lead is up to 18.28 AP on MS-COCO). Our code is available at \url{https://github.com/chuxiuhong/DualTeacher}.

Figures (8)

• Figure 1: Comparison of IOD and SSIOD.
• Figure 2: Comparison of object detection results for different teacher models.
• Figure 3: Comparison of representative methods for IOD (gray dashed arrow) and our DualTeacher for SSIOD (blue dashed arrow). The black dashed arrow denotes the general framework of semi-supervised learning for object detection. The locked and shaded areas indicate that the parameters are fixed.
• Figure 4: Performance of SSIOD. Here we take Faster ILOD as the baseline approach of knowledge distillation (KD), which can be implemented to $f_{{\rm{tea}}}$, $f_{{\rm{stu}}}$, or both.
• Figure 5: Subsequent performance of the classes learned in the initial phase. The number of new classes is balanced by task. "-R" stands for replay of all labelled data.