Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Out-of-Distribution Detection Using Peer-Class Generated by Large Language Model

K Huang, G Song, Hanwen Su, Jiyan Wang

TL;DR

A novel method called ODPC is proposed, in which specific prompts to generate OOD peer classes of ID semantics are designed by a large language model as an auxiliary modality to facilitate detection.

Abstract

Out-of-distribution (OOD) detection is a critical task to ensure the reliability and security of machine learning models deployed in real-world applications. Conventional methods for OOD detection that rely on single-modal information, often struggle to capture the rich variety of OOD instances. The primary difficulty in OOD detection arises when an input image has numerous similarities to a particular class in the in-distribution (ID) dataset, e.g., wolf to dog, causing the model to misclassify it. Nevertheless, it may be easy to distinguish these classes in the semantic domain. To this end, in this paper, a novel method called ODPC is proposed, in which specific prompts to generate OOD peer classes of ID semantics are designed by a large language model as an auxiliary modality to facilitate detection. Moreover, a contrastive loss based on OOD peer classes is devised to learn compact representations of ID classes and improve the clarity of boundaries between different classes. The extensive experiments on five benchmark datasets show that the method we propose can yield state-of-the-art results.

Out-of-Distribution Detection Using Peer-Class Generated by Large Language Model

TL;DR

A novel method called ODPC is proposed, in which specific prompts to generate OOD peer classes of ID semantics are designed by a large language model as an auxiliary modality to facilitate detection.

Abstract

Out-of-distribution (OOD) detection is a critical task to ensure the reliability and security of machine learning models deployed in real-world applications. Conventional methods for OOD detection that rely on single-modal information, often struggle to capture the rich variety of OOD instances. The primary difficulty in OOD detection arises when an input image has numerous similarities to a particular class in the in-distribution (ID) dataset, e.g., wolf to dog, causing the model to misclassify it. Nevertheless, it may be easy to distinguish these classes in the semantic domain. To this end, in this paper, a novel method called ODPC is proposed, in which specific prompts to generate OOD peer classes of ID semantics are designed by a large language model as an auxiliary modality to facilitate detection. Moreover, a contrastive loss based on OOD peer classes is devised to learn compact representations of ID classes and improve the clarity of boundaries between different classes. The extensive experiments on five benchmark datasets show that the method we propose can yield state-of-the-art results.
Paper Structure (15 sections, 3 equations, 3 figures, 2 tables)

This paper contains 15 sections, 3 equations, 3 figures, 2 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Out-of-distribution Detection with Single-modal
  4. Out-of-distribution Detection with Multi-modal
  5. Method
  6. The Architecture
  7. Peer-Class Generation by LLM
  8. Inter-Modal OOD Generation
  9. Training with Multi-Modal
  10. KNN for inference
  11. Experiments
  12. Implementation and Settings
  13. Results and Discussion
  14. Ablation Study
  15. Conclusion

Figures (3)

  • Figure 1: In subfigure (a), the two modalities of the ID categories exhibit semantic similarity, while the orange rectangle indicates the unavailable of the OOD image modality, the orange circle signifies the available of the OOD text modality. The subfigure (b) illustrates how LLM generates peer-class labels based on ID class.
  • Figure 2: This diagram illustrates the framework of our approach. First, we use LLM to generate peer-class labels based on in-distribution (ID) classes. Then, these peer-class labels, along with the ID labels, are processed by the text encoder to extract text features, while the ID images are subject to feature extraction using the image encoder. Within the MLP, negative examples are generated by the mixup method and the Peer-Class Contrastive loss is used on each fully connected layer, the cross-entropy loss is added in the classification layer. Finally, we use KNN method to inference by using image features from MLP.
  • Figure 3: The green dots means ID features and the pink dots means OOD features. After training, the features of the penultimate layer reveal that the characteristics of OOD have diverged from the clusters of the ID dataset.