OOVDet: Low-Density Prior Learning for Zero-Shot Out-of-Vocabulary Object Detection

TL;DR

The paper defines zero-shot out-of-vocabulary detection (ZS-OOVD) and introduces OOVDet, a framework that explicitly models IV–OOV boundaries in open-world scenes. It combines three components: Out-of-Vocabulary Prompt Synthesis (OPS) to generate low-density OOV prompts in latent space, Dirichlet-based Gradient Attribution (DGA) to mine high-uncertainty pseudo-OOV images, and Low-Density Prior Constraint (LPC) to regularize OOV optimization via Gaussian-kernel density estimation. These modules together construct a discriminative IV–OOV boundary and improve OOV recall while preserving IV accuracy, achieving substantial gains on OOV-VOC and OOV-COCO benchmarks and showing robust generalization to real-world open-world data. The approach demonstrates that integrating language-aware prompts, uncertainty-driven pseudo-samples, and density-aware priors yields reliable OOV rejection in zero-shot settings, with practical implications for safe and open-world perception systems. The authors also release a ZS-OOVD benchmark and show that their method reduces open-set errors and improves localization of unknown objects across diverse domains.

Abstract

Zero-shot out-of-vocabulary detection (ZS-OOVD) aims to accurately recognize objects of in-vocabulary (IV) categories provided at zero-shot inference, while simultaneously rejecting undefined ones (out-of-vocabulary, OOV) that lack corresponding category prompts. However, previous methods are prone to overfitting the IV classes, leading to the OOV or undefined classes being misclassified as IV ones with a high confidence score. To address this issue, this paper proposes a zero-shot OOV detector (OOVDet), a novel framework that effectively detects predefined classes while reliably rejecting undefined ones in zero-shot scenes. Specifically, due to the model's lack of prior knowledge about the distribution of OOV data, we synthesize region-level OOV prompts by sampling from the low-likelihood regions of the class-conditional Gaussian distributions in the hidden space, motivated by the assumption that unknown semantics are more likely to emerge in low-density areas of the latent space. For OOV images, we further propose a Dirichlet-based gradient attribution mechanism to mine pseudo-OOV image samples, where the attribution gradients are interpreted as Dirichlet evidence to estimate prediction uncertainty, and samples with high uncertainty are selected as pseudo-OOV images. Building on these synthesized OOV prompts and pseudo-OOV images, we construct the OOV decision boundary through a low-density prior constraint, which regularizes the optimization of OOV classes using Gaussian kernel density estimation in accordance with the above assumption. Experimental results show that our method significantly improves the OOV detection performance in zero-shot scenes. The code is available at https://github.com/binyisu/OOV-detector.

Figures (9)

• Figure 1: The zero-shot object detection (ZSOD) task vs the zero-shot out-of-vocabulary detection (ZS-OOVD) task.
• Figure 2: The framework of our OOVDet for the challenging zero-shot out-of-vocabulary object detection task. OOVDet is a simple two-stage detector with (a) a novel Out-of-vocabulary Prompt Synthesis (OPS), (b) a novel Dirichlet-based Gradient Attribution (DGA) for pseudo-unknowns mining and (c) a novel Low-density Prior Constraint (LPC) for OOV optimization.
• Figure 3: Problem Setup of ZS-OOVD.
• Figure 4: Effect of kernel bandwidth $h$ on IV, unseen-IV, and OOV detection performance. We analyze the kernel bandwidth $h$ in the Low-Density Prior Constraint on OOV-VOC and OOV-COCO, showing that $h$ strongly affects both IV and OOV performance. A moderate bandwidth ($h=0.8$) provides the best trade-off, maximizing IV accuracy while preserving unseen-IV generalization and stable OOV detection. The blue and red lines correspond to OOV-VOC and OOV-COCO evaluations, respectively.
• Figure 5: The visualized results on OOV-VOC,OOV-COCO and diverse real-world datasets.