On the Inherent Robustness of One-Stage Object Detection against Out-of-Distribution Data

TL;DR

This paper addresses the challenge of detecting unknown objects in open-world scenarios using pretrained one-stage object detectors without retraining. It introduces FMap, a feature-map–based OoD detector that uses per-stride embeddings and centroid distances, and extends it with Supervised Dimensionality Reduction (IVIS) and Enhanced Unknown Localization (EUL) to boost unknown recall. The authors compare FMap and its variants to logits-based post-hoc OoD methods and state-of-the-art OWOD approaches on the Unknown Object Detection benchmark, showing competitive or superior performance while preserving known-object accuracy and avoiding retraining. A key insight is that fusion of feature-based and logits-based detectors yields the strongest robustness, surpassing individual methods in the open-world setting. The work highlights a practical, scalable path for deploying pretrained detectors with solid OoD detection capabilities.

Abstract

Robustness is a fundamental aspect for developing safe and trustworthy models, particularly when they are deployed in the open world. In this work we analyze the inherent capability of one-stage object detectors to robustly operate in the presence of out-of-distribution (OoD) data. Specifically, we propose a novel detection algorithm for detecting unknown objects in image data, which leverages the features extracted by the model from each sample. Differently from other recent approaches in the literature, our proposal does not require retraining the object detector, thereby allowing for the use of pretrained models. Our proposed OoD detector exploits the application of supervised dimensionality reduction techniques to mitigate the effects of the curse of dimensionality on the features extracted by the model. Furthermore, it utilizes high-resolution feature maps to identify potential unknown objects in an unsupervised fashion. Our experiments analyze the Pareto trade-off between the performance detecting known and unknown objects resulting from different algorithmic configurations and inference confidence thresholds. We also compare the performance of our proposed algorithm to that of logits-based post-hoc OoD methods, as well as possible fusion strategies. Finally, we discuss on the competitiveness of all tested methods against state-of-the-art OoD approaches for object detection models over the recently published Unknown Object Detection benchmark. The obtained results verify that the performance of avant-garde post-hoc OoD detectors can be further improved when combined with our proposed algorithm.

Figures (13)

• Figure 1: Block diagram describing the general operation of the proposed FMap detector. In the inference part, an example for the stride ($\varphi$) and class ($c$) values is displayed.
• Figure 2: Graphical example of the application of the Enhanced Unknown Localization (EUL) algorithm. Red boxes in step ④ are bounding box proposals. In the final image after step ⑤, the orange box is an unknown object detected by the basic FMap algorithm, whereas yellow are the boxes selected by EUL, and finally the pink boxes are the unknown object annotations.
• Figure 3: Illustration of the AND and OR fusion strategies. A decision of '1' indicates that the method identifies the prediction as ID, whereas '0' indicates the contrary.
• Figure 4: Example of the application of the SCORE strategy. Each method computes a fusion score for $f_{\mathit{fusion}}(\cdot)$ using the piece-wise functions depicted in green and red. Values outside the defined range are clipped to the minimum and maximum values, respectively.
• Figure 5: Front of non-dominated FMap configurations in the mAP versus U-F1$_{\mathit{SUM}}$ trade-off for the vanilla version of FMap. Points correspond to different configurations of the model in terms of distance metrics (represented by marker types) and clustering methods (indicated by color). For each configuration, various inference confidence thresholds are represented. We refer to Subsection \ref{['ssec:owod_results_preamble']} for further details.