Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Large Language Models for Anomaly and Out-of-Distribution Detection: A Survey

Ruiyao Xu, Kaize Ding

TL;DR

This survey addresses anomaly and out-of-distribution detection in the era of Large Language Models (LLMs) by introducing a two-fold taxonomy: using LLMs for detection (prompting-based and contrasting-based) and using LLMs for generation (augmentation and explanations). It systematically reviews methods across modalities, discusses PEFT and prompt-tuning strategies to adapt LLMs, and highlights the emergence of multimodal LLMs in this domain. The authors catalog datasets, compare performance across settings, and identify challenges such as explainability, hallucination, and efficiency, while outlining future directions like integrating domain knowledge and robust multimodal probing. The work underscores the potential of LLMs to provide zero-shot or few-shot detection capabilities, enhanced interpretability, and scalable data augmentation for robust anomaly and OOD detection in real-world deployments.

Abstract

Detecting anomalies or out-of-distribution (OOD) samples is critical for maintaining the reliability and trustworthiness of machine learning systems. Recently, Large Language Models (LLMs) have demonstrated their effectiveness not only in natural language processing but also in broader applications due to their advanced comprehension and generative capabilities. The integration of LLMs into anomaly and OOD detection marks a significant shift from the traditional paradigm in the field. This survey focuses on the problem of anomaly and OOD detection under the context of LLMs. We propose a new taxonomy to categorize existing approaches into two classes based on the role played by LLMs. Following our proposed taxonomy, we further discuss the related work under each of the categories and finally discuss potential challenges and directions for future research in this field. We also provide an up-to-date reading list of relevant papers.

Large Language Models for Anomaly and Out-of-Distribution Detection: A Survey

TL;DR

This survey addresses anomaly and out-of-distribution detection in the era of Large Language Models (LLMs) by introducing a two-fold taxonomy: using LLMs for detection (prompting-based and contrasting-based) and using LLMs for generation (augmentation and explanations). It systematically reviews methods across modalities, discusses PEFT and prompt-tuning strategies to adapt LLMs, and highlights the emergence of multimodal LLMs in this domain. The authors catalog datasets, compare performance across settings, and identify challenges such as explainability, hallucination, and efficiency, while outlining future directions like integrating domain knowledge and robust multimodal probing. The work underscores the potential of LLMs to provide zero-shot or few-shot detection capabilities, enhanced interpretability, and scalable data augmentation for robust anomaly and OOD detection in real-world deployments.

Abstract

Detecting anomalies or out-of-distribution (OOD) samples is critical for maintaining the reliability and trustworthiness of machine learning systems. Recently, Large Language Models (LLMs) have demonstrated their effectiveness not only in natural language processing but also in broader applications due to their advanced comprehension and generative capabilities. The integration of LLMs into anomaly and OOD detection marks a significant shift from the traditional paradigm in the field. This survey focuses on the problem of anomaly and OOD detection under the context of LLMs. We propose a new taxonomy to categorize existing approaches into two classes based on the role played by LLMs. Following our proposed taxonomy, we further discuss the related work under each of the categories and finally discuss potential challenges and directions for future research in this field. We also provide an up-to-date reading list of relevant papers.
Paper Structure (39 sections, 6 equations, 4 figures, 3 tables)

This paper contains 39 sections, 6 equations, 4 figures, 3 tables.

Table of Contents

  1. Introduction
  2. Preliminaries
  3. Problem Definition
  4. LLMs for Detection
  5. Prompting-based Detection
  6. Detection without LLM Tuning
  7. Detection with LLM Tuning
  8. Contrasting-based Detection
  9. Detection without LLM Tuning
  10. Detection with LLM Tuning
  11. LLMs for Generation
  12. Augmentation-centric Generation
  13. Text Embedding-based Augmentation
  14. Pseudo Label-based Augmentation
  15. Textual Description-based Augmentation
  16. ...and 24 more sections

Figures (4)

  • Figure 1: A simple illustration of leveraging LLMs for vision anomaly and OOD detection.
  • Figure 2: Taxonomy of methods utilizing LLMs for anomaly and OOD detection tasks.
  • Figure 3: The illustration of two approaches in (§ \ref{['sec:detection']}): (a) Prompting-based Detection and (b) Contrasting-based Detection.
  • Figure 4: The illustration of four approaches in (§ \ref{['sec:generation']}): (a) Text Embedding-based Augmentation; (b) Pseudo Label-based Augmentation; (c) Textual Description-based Augmentation; and (d) Explanation-centric Generation.

Theorems & Definitions (2)

  • Definition 1
  • Definition 2