Generalized Open-World Semi-Supervised Object Detection

TL;DR

An ensemble-based OOD Explorer for detection and classification, and an adaptable semi-supervised object detection framework that integrates both ID and OOD data are introduced that performs competitively against state-of-the-art OOD detection algorithms and also significantly boosts the semi-supervised learning performance for both ID and OOD classes.

Abstract

Traditional semi-supervised object detection methods assume a fixed set of object classes (in-distribution or ID classes) during training and deployment, which limits performance in real-world scenarios where unseen classes (out-of-distribution or OOD classes) may appear. In such cases, OOD data is often misclassified as ID, thus harming the ID classes accuracy. Open-set methods address this limitation by filtering OOD data to improve ID performance, thereby limiting the learning process to ID classes. We extend this to a more natural open-world setting, where the OOD classes are not only detected but also incorporated into the learning process. Specifically, we explore two key questions: 1) how to accurately detect OOD samples, and, most importantly, 2) how to effectively learn from the OOD samples in a semi-supervised object detection pipeline without compromising ID accuracy. To address this, we introduce an ensemble-based OOD Explorer for detection and classification, and an adaptable semi-supervised object detection framework that integrates both ID and OOD data. Through extensive evaluation on different open-world scenarios, we demonstrate that our method performs competitively against state-of-the-art OOD detection algorithms and also significantly boosts the semi-supervised learning performance for both ID and OOD classes.

Figures (8)

• Figure 1: In our task of generalized open-world semi-supervised object detection (OWSSD), the model observes a labeled dataset with ID data and an unlabeled dataset which can contain both ID and OOD data. The objective is to train a detection model that is able to (1) localize and classify instances belonging to seen classes; and (2) localize instances not belonging to seen classes and group them into a new 'unknown' class.
• Figure 2: A conceptual overview of our OOD-aware semi-supervised learning framework. In the first stage, a Teacher Model is trained from the available labeled images with ID categories. In the second stage, unlabeled images that could contain ID and OOD categories are passed through (1) the trained Teacher Model to generate pseudo-labels for the ID categories that the Teacher model was trained on; (2) an OOD Detector to discover any novel OOD categories present in the unlabeled data. The pseudo-labels generated by the two modules are then fused such that in cases of conflict (high overlap), the novel OOD labels get preference. This helps eliminate false negatives that could be introduced when an OOD category is wrongly classified as an ID category. Finally, a Student model is trained jointly on the labeled and unlabeled data to minimize the total loss.
• Figure 3: Training Stage
• Figure 4: Testing Stage
• Figure 6: Qualitative results using our OOD Detector and CutLER (Cascade) wang2023cut as the open world localization method.