Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Graph Pre-Training Models Are Strong Anomaly Detectors

Jiashun Cheng, Zinan Zheng, Yang Liu, Jianheng Tang, Hongwei Wang, Yu Rong, Jia Li, Fugee Tsung

TL;DR

It is demonstrated that pre-training is highly competitive, markedly outperforming the state-of-the-art end-to-end training models when faced with limited supervision, and enhances the detection of distant, under-represented, unlabeled anomalies that go beyond 2-hop neighborhoods of known anomalies.

Abstract

Graph Anomaly Detection (GAD) is a challenging and practical research topic where Graph Neural Networks (GNNs) have recently shown promising results. The effectiveness of existing GNNs in GAD has been mainly attributed to the simultaneous learning of node representations and the classifier in an end-to-end manner. Meanwhile, graph pre-training, the two-stage learning paradigm such as DGI and GraphMAE, has shown potential in leveraging unlabeled graph data to enhance downstream tasks, yet its impact on GAD remains under-explored. In this work, we show that graph pre-training models are strong graph anomaly detectors. Specifically, we demonstrate that pre-training is highly competitive, markedly outperforming the state-of-the-art end-to-end training models when faced with limited supervision. To understand this phenomenon, we further uncover pre-training enhances the detection of distant, under-represented, unlabeled anomalies that go beyond 2-hop neighborhoods of known anomalies, shedding light on its superior performance against end-to-end models. Moreover, we extend our examination to the potential of pre-training in graph-level anomaly detection. We envision this work to stimulate a re-evaluation of pre-training's role in GAD and offer valuable insights for future research.

Graph Pre-Training Models Are Strong Anomaly Detectors

TL;DR

It is demonstrated that pre-training is highly competitive, markedly outperforming the state-of-the-art end-to-end training models when faced with limited supervision, and enhances the detection of distant, under-represented, unlabeled anomalies that go beyond 2-hop neighborhoods of known anomalies.

Abstract

Graph Anomaly Detection (GAD) is a challenging and practical research topic where Graph Neural Networks (GNNs) have recently shown promising results. The effectiveness of existing GNNs in GAD has been mainly attributed to the simultaneous learning of node representations and the classifier in an end-to-end manner. Meanwhile, graph pre-training, the two-stage learning paradigm such as DGI and GraphMAE, has shown potential in leveraging unlabeled graph data to enhance downstream tasks, yet its impact on GAD remains under-explored. In this work, we show that graph pre-training models are strong graph anomaly detectors. Specifically, we demonstrate that pre-training is highly competitive, markedly outperforming the state-of-the-art end-to-end training models when faced with limited supervision. To understand this phenomenon, we further uncover pre-training enhances the detection of distant, under-represented, unlabeled anomalies that go beyond 2-hop neighborhoods of known anomalies, shedding light on its superior performance against end-to-end models. Moreover, we extend our examination to the potential of pre-training in graph-level anomaly detection. We envision this work to stimulate a re-evaluation of pre-training's role in GAD and offer valuable insights for future research.

Paper Structure

This paper contains 16 sections, 2 equations, 5 figures, 11 tables.

Table of Contents

  1. Introduction
  2. Related Works
  3. Graph Anomaly Detection
  4. Graph Pre-Training
  5. Preliminaries
  6. Experimental Results
  7. Configurations
  8. When Pre-Training Excels in Graph Anomaly Detection
  9. Why Pre-Training Excels in Graph Anomaly Detection
  10. The Impact of Graph Sparsity
  11. From Sparsity to Reachable Ratio
  12. Pre-training Improves the Detection of 2-hop Unreachable Anomalies
  13. Key Factors Influencing the Success of Pre-training
  14. Potential of Pre-Training in Graph-Level Anomaly Detection
  15. Conclusion
  16. ...and 1 more sections

Figures (5)

  • Figure 1: The k-hop reachable ratio $R_{k}$ till 3-hop neighborhood across all datasets.
  • Figure 2: The average AUROC score of DGI in downstream anomaly detection with different perturbation ratios used in the pre-training stage.
  • Figure 3: The 2-hop reachable ratio $R_{2}$ with different number of labeled anomalies across all datasets.
  • Figure 4: The average ranking for unlabeled anomalies located in the k$^{th}$-hop neighborhood.
  • Figure 5: The average AUROC score with the different number of labeled anomalies.

Theorems & Definitions (1)

  • Definition 1: K-hop reachable ratio