Dream-Box: Object-wise Outlier Generation for Out-of-Distribution Detection

TL;DR

Out-of-distribution detection is essential for robust deployment when test data differ from training distributions, especially for object-level tasks. Dream-Box introduces object-wise pixel-space outlier generation using diffusion models to synthesize OOD objects within image regions, paired with an energy-based detector head to separate in-distribution and OOD. The method uses two prompt strategies (generic prompts and distance-based prompts) and trains on VOC in-distribution with COCO as OOD, achieving competitive results with traditional OOD approaches and enabling interpretable visualization of OOD objects via pixel-space generation. The work demonstrates the value of visualizable outliers for diagnosing OOD failures and suggests directions to improve prompt strategies and extend to other vision tasks.

Abstract

Deep neural networks have demonstrated great generalization capabilities for tasks whose training and test sets are drawn from the same distribution. Nevertheless, out-of-distribution (OOD) detection remains a challenging task that has received significant attention in recent years. Specifically, OOD detection refers to the detection of instances that do not belong to the training distribution, while still having good performance on the in-distribution task (e.g., classification or object detection). Recent work has focused on generating synthetic outliers and using them to train an outlier detector, generally achieving improved OOD detection than traditional OOD methods. In this regard, outliers can be generated either in feature or pixel space. Feature space driven methods have shown strong performance on both the classification and object detection tasks, at the expense that the visualization of training outliers remains unknown, making further analysis on OOD failure modes challenging. On the other hand, pixel space outlier generation techniques enabled by diffusion models have been used for image classification using, providing improved OOD detection performance and outlier visualization, although their adaption to the object detection task is as yet unexplored. We therefore introduce Dream-Box, a method that provides a link to object-wise outlier generation in the pixel space for OOD detection. Specifically, we use diffusion models to generate object-wise outliers that are used to train an object detector for an in-distribution task and OOD detection. Our method achieves comparable performance to previous traditional methods while being the first technique to provide concrete visualization of generated OOD objects.

Figures (6)

• Figure 1: Dream-Box enables object-wise OOD detection by generating objects using embeddings far from the class-name text embeddings.
• Figure 2: Dream-Box overview. We generate outlier objects using two prompt strategies and leveraging in-distribution objects. Subsequently, we train an object detector with a classification output head for in-distribution/OOD.
• Figure 3: Object detector with modified OOD output (head).
• Figure 4: Performance of the distance-based modified prompting strategy with different $\sigma$ values.
• Figure 5: Exemplar generated outlier objects of the generic prompt and distance-based modified prompting strategies.