Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

AnomalyLLM: Few-shot Anomaly Edge Detection for Dynamic Graphs using Large Language Models

Shuo Liu, Di Yao, Lanting Fang, Zhetao Li, Wenbin Li, Kaiyu Feng, XiaoWen Ji, Jingping Bi

TL;DR

AnomalyLLM pretrains a dynamic-aware encoder to generate the representations of edges and reprograms the edges using the prototypes of word embeddings and design an in-context learning framework that integrates the information of a few labeled samples to achieve few-shot anomaly detection.

Abstract

Detecting anomaly edges for dynamic graphs aims to identify edges significantly deviating from the normal pattern and can be applied in various domains, such as cybersecurity, financial transactions and AIOps. With the evolving of time, the types of anomaly edges are emerging and the labeled anomaly samples are few for each type. Current methods are either designed to detect randomly inserted edges or require sufficient labeled data for model training, which harms their applicability for real-world applications. In this paper, we study this problem by cooperating with the rich knowledge encoded in large language models(LLMs) and propose a method, namely AnomalyLLM. To align the dynamic graph with LLMs, AnomalyLLM pre-trains a dynamic-aware encoder to generate the representations of edges and reprograms the edges using the prototypes of word embeddings. Along with the encoder, we design an in-context learning framework that integrates the information of a few labeled samples to achieve few-shot anomaly detection. Experiments on four datasets reveal that AnomalyLLM can not only significantly improve the performance of few-shot anomaly detection, but also achieve superior results on new anomalies without any update of model parameters.

AnomalyLLM: Few-shot Anomaly Edge Detection for Dynamic Graphs using Large Language Models

TL;DR

AnomalyLLM pretrains a dynamic-aware encoder to generate the representations of edges and reprograms the edges using the prototypes of word embeddings and design an in-context learning framework that integrates the information of a few labeled samples to achieve few-shot anomaly detection.

Abstract

Detecting anomaly edges for dynamic graphs aims to identify edges significantly deviating from the normal pattern and can be applied in various domains, such as cybersecurity, financial transactions and AIOps. With the evolving of time, the types of anomaly edges are emerging and the labeled anomaly samples are few for each type. Current methods are either designed to detect randomly inserted edges or require sufficient labeled data for model training, which harms their applicability for real-world applications. In this paper, we study this problem by cooperating with the rich knowledge encoded in large language models(LLMs) and propose a method, namely AnomalyLLM. To align the dynamic graph with LLMs, AnomalyLLM pre-trains a dynamic-aware encoder to generate the representations of edges and reprograms the edges using the prototypes of word embeddings. Along with the encoder, we design an in-context learning framework that integrates the information of a few labeled samples to achieve few-shot anomaly detection. Experiments on four datasets reveal that AnomalyLLM can not only significantly improve the performance of few-shot anomaly detection, but also achieve superior results on new anomalies without any update of model parameters.
Paper Structure (39 sections, 14 equations, 6 figures, 7 tables)

This paper contains 39 sections, 14 equations, 6 figures, 7 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Preliminary
  4. Problem Definition
  5. Overview of AnomalyLLM
  6. Methodology
  7. Dynamic-aware Contrastive Pretraining
  8. Dynamic-aware Encoder
  9. Contrastive Learning-based Optimization
  10. Reprogramming-based Modality Alignment
  11. Text Prototype Reprogramming
  12. Pseudo Label for Anomaly Fine-tuning
  13. In-Context Learning for Few-Shot Detection
  14. Prompt Template Construction
  15. Few-shot anomaly detection
  16. ...and 24 more sections

Figures (6)

  • Figure 1: The motivation of AnomalyLLM. In the real world, edge anomaly types are diverse, evolving over time, and typically associated with limited labeled data.
  • Figure 2: Overview of AnomalyLLM. AnomalyLLM comprises three modules: Dynamic-aware Contrastive Pretraining, Reprogramming-based Modality Alignment, and In-Context Learning for Few-Shot Detection.
  • Figure 3: Sample process of contrastive training triplet
  • Figure 4: The prompt of In-Context Learning
  • Figure 5: Inference time of AnomalyLLM
  • ...and 1 more figures