Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Learning Transferable Negative Prompts for Out-of-Distribution Detection

Tianqi Li, Guansong Pang, Xiao Bai, Wenjun Miao, Jin Zheng

TL;DR

The paper addresses the challenge of detecting Out-of-Distribution (OOD) samples when target datasets lack OOD images and class coverage, by introducing NegPrompt, a CLIP-based approach that learns negative prompts tied to in-distribution (ID) classes. NegPrompt trains a small set of negative prompts using only ID data and leverages the transferability of these prompts to handle novel IDs in open-vocabulary scenarios, eliminating the need for external outlier data. The method employs a three-loss objective and a two-stage training process to craft diverse, informative negative semantics that separate ID from OOD data and integrates them into an extended MCM-style scoring at inference. Across ImageNet-based benchmarks, NegPrompt achieves state-of-the-art performance on conventional, hard, and open-vocabulary OOD detection while preserving ID classification accuracy, underscoring its practical value for robust visual recognition systems.

Abstract

Existing prompt learning methods have shown certain capabilities in Out-of-Distribution (OOD) detection, but the lack of OOD images in the target dataset in their training can lead to mismatches between OOD images and In-Distribution (ID) categories, resulting in a high false positive rate. To address this issue, we introduce a novel OOD detection method, named 'NegPrompt', to learn a set of negative prompts, each representing a negative connotation of a given class label, for delineating the boundaries between ID and OOD images. It learns such negative prompts with ID data only, without any reliance on external outlier data. Further, current methods assume the availability of samples of all ID classes, rendering them ineffective in open-vocabulary learning scenarios where the inference stage can contain novel ID classes not present during training. In contrast, our learned negative prompts are transferable to novel class labels. Experiments on various ImageNet benchmarks show that NegPrompt surpasses state-of-the-art prompt-learning-based OOD detection methods and maintains a consistent lead in hard OOD detection in closed- and open-vocabulary classification scenarios. Code is available at https://github.com/mala-lab/negprompt.

Learning Transferable Negative Prompts for Out-of-Distribution Detection

TL;DR

The paper addresses the challenge of detecting Out-of-Distribution (OOD) samples when target datasets lack OOD images and class coverage, by introducing NegPrompt, a CLIP-based approach that learns negative prompts tied to in-distribution (ID) classes. NegPrompt trains a small set of negative prompts using only ID data and leverages the transferability of these prompts to handle novel IDs in open-vocabulary scenarios, eliminating the need for external outlier data. The method employs a three-loss objective and a two-stage training process to craft diverse, informative negative semantics that separate ID from OOD data and integrates them into an extended MCM-style scoring at inference. Across ImageNet-based benchmarks, NegPrompt achieves state-of-the-art performance on conventional, hard, and open-vocabulary OOD detection while preserving ID classification accuracy, underscoring its practical value for robust visual recognition systems.

Abstract

Existing prompt learning methods have shown certain capabilities in Out-of-Distribution (OOD) detection, but the lack of OOD images in the target dataset in their training can lead to mismatches between OOD images and In-Distribution (ID) categories, resulting in a high false positive rate. To address this issue, we introduce a novel OOD detection method, named 'NegPrompt', to learn a set of negative prompts, each representing a negative connotation of a given class label, for delineating the boundaries between ID and OOD images. It learns such negative prompts with ID data only, without any reliance on external outlier data. Further, current methods assume the availability of samples of all ID classes, rendering them ineffective in open-vocabulary learning scenarios where the inference stage can contain novel ID classes not present during training. In contrast, our learned negative prompts are transferable to novel class labels. Experiments on various ImageNet benchmarks show that NegPrompt surpasses state-of-the-art prompt-learning-based OOD detection methods and maintains a consistent lead in hard OOD detection in closed- and open-vocabulary classification scenarios. Code is available at https://github.com/mala-lab/negprompt.
Paper Structure (24 sections, 8 equations, 4 figures, 8 tables)

This paper contains 24 sections, 8 equations, 4 figures, 8 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Pre-trained Vision-Language Models
  4. Prompt Learning
  5. Out-of-Distribution Detection
  6. Method
  7. Preliminaries
  8. Proposed Approach
  9. Learning Negative Prompts
  10. Open-Vocabulary Capability
  11. Inference
  12. Experiment
  13. Experimental Details
  14. Comparison to State-of-the-art Models
  15. Classification Accuracy on ID Data
  16. ...and 9 more sections

Figures (4)

  • Figure 1: Illustration of the key intuition of NegPrompt. For each ID class, NegPrompt trains a small set of learnable prompts that have negative semantics to the learned positive prompt of the given class. As a result, OOD samples exhibit higher similarity to the negative prompts than the positive prompts.
  • Figure 2: Overview of NegPropmt. Given the CLIP model and positive prompts learned by existing prompt learning methods such as CoOp zhou2022learning, NegPrompt learns a set of negative prompts relative to different ID class labels via three loss functions that enforce the separation between negative prompts and ID images, and between negative and positive prompts, as well as the diversity of the negative prompts.
  • Figure 3: Similarity of ID/OOD and Positive/Negative Prompts.
  • Figure 4: T-SNE visualization of NegPrompt, utilizing a subset of ImageNet - TinyImageNet as the dataset.