Tunable Hybrid Proposal Networks for the Open World

TL;DR

This work tackles open-world object proposal by introducing Tunable Hybrid Proposal Network (THPN), a two-stage, class-agnostic proposal system that blends classification-based and localization-based objectness via a tunable parameter $\lambda_{CLS}$. It couples a self-training procedure with dynamic loss terms (Localization Quality Focal Loss and Weighted Box Regression) to robustly handle biased labeling and imperfect pseudo-labels, enabling strong performance across ID and OOD recalls. THPN demonstrates superior recall over baselines (RPN, OLN) across COCO-based open-world benchmarks and in diverse challenges such as training-class diversity, semi-supervised labeling, and remote-sensing ships, while remaining data-efficient. The method’s flexible attribution between ID and OOD detection makes it practical for real-world deployment where task requirements vary, and its open-world evaluation protocol provides a rigorous benchmark for future work. Overall, THPN advances open-world detection by delivering a tunable, data-efficient proposal mechanism that improves recall for both known and novel objects.

Abstract

Current state-of-the-art object proposal networks are trained with a closed-world assumption, meaning they learn to only detect objects of the training classes. These models fail to provide high recall in open-world environments where important novel objects may be encountered. While a handful of recent works attempt to tackle this problem, they fail to consider that the optimal behavior of a proposal network can vary significantly depending on the data and application. Our goal is to provide a flexible proposal solution that can be easily tuned to suit a variety of open-world settings. To this end, we design a Tunable Hybrid Proposal Network (THPN) that leverages an adjustable hybrid architecture, a novel self-training procedure, and dynamic loss components to optimize the tradeoff between known and unknown object detection performance. To thoroughly evaluate our method, we devise several new challenges which invoke varying degrees of label bias by altering known class diversity and label count. We find that in every task, THPN easily outperforms existing baselines (e.g., RPN, OLN). Our method is also highly data efficient, surpassing baseline recall with a fraction of the labeled data.

Figures (9)

• Figure 1: THPN's adjustability allows it to excel across a broad range of open world tasks. Top: optimal $\lambda_{CLS}$ for each task. Bottom: recall comparison; annotations are THPN's margins over OLN OLN.
• Figure 2: Overview of THPN's architecture and training procedure.
• Figure 3: THPN training samples ($p$=30%). Blue boxes are ID labels and cyan boxes are pseudo-labels with objectness $s\in[0,1]$.
• Figure 4: Hyperparameter $p$'s impact on ALL recall and pseudo-label count for a THPN trained on the VOC split.
• Figure 5: Differences between Faster R-CNN, OLN, and our hybrid THPN architecture.