O1O: Grouping of Known Classes to Identify Unknown Objects as Odd-One-Out

TL;DR

Drawing inspiration from studies on human cognition, it is found that geometric cues improve unknown recall, and by identifying similarities between classes within a superclass, can identify unknown classes through an odd-one-out scoring mechanism.

Abstract

Object detection methods trained on a fixed set of known classes struggle to detect objects of unknown classes in the open-world setting. Current fixes involve adding approximate supervision with pseudo-labels corresponding to candidate locations of objects, typically obtained in a class-agnostic manner. While previous approaches mainly rely on the appearance of objects, we find that geometric cues improve unknown recall. Although additional supervision from pseudo-labels helps to detect unknown objects, it also introduces confusion for known classes. We observed a notable decline in the model's performance for detecting known objects in the presence of noisy pseudo-labels. Drawing inspiration from studies on human cognition, we propose to group known classes into superclasses. By identifying similarities between classes within a superclass, we can identify unknown classes through an odd-one-out scoring mechanism. Our experiments on open-world detection benchmarks demonstrate significant improvements in unknown recall, consistently across all tasks. Crucially, we achieve this without compromising known performance, thanks to better partitioning of the feature space with superclasses.

Figures (12)

• Figure 1: Superclass Prior vs. Single Objectness Distribution. This figure compares the projections of object queries trained with superclass prior in our approach O1O (left) vs. fitting a single objectness distribution to matched queries as in PROB Zohar2023CVPR (right) for S-OWOD Task 4. O1O shapes the representation space by encouraging queries of similar classes to group together, allowing us to identify odd-one-out queries as unknown objects.
• Figure 1: Superclass Separation Across Tasks.
• Figure 1: Number of Pseudo-labels Ablation.
• Figure 2: Overview of Proposed O1O. Building on Deformable DETR Zhu2021ICLR(blue), we first add supervision for unknowns with geometric pseudo-labels. Following GOOD Huang2023ICLR, we extract pseudo-labels (dashed) from a Region Proposal Network (RPN) trained on surface normal maps (green). This allows us to localize unknown objects based on geometric cues in a class-agnostic manner. Noisy pseudo-labels tend to hurt the known performance of a model. To mitigate this issue, we propose to group queries into superclasses with a superclass head (red). By incorporating the learned prior from superclasses into the scoring function, we achieve the best balance between known and unknown performance.
• Figure 2: Inference Score Ablation. We perform an ablation study on the inference score used for unknowns on S-OWOD Task 1. The first row corresponds to MSP Hendrycks2017ICLR applied to superclass probabilities, and the second row is MSP applied to recalibrated class probabilities $p'$\ref{['eq:score']}. The third row is an extended version of MSP, which uses the union of recalibrated class probabilities $p'$. As shown in the last row, we use the third with a threshold that is experimentally set on the training set.