CVTGAD: Simplified Transformer with Cross-View Attention for Unsupervised Graph-level Anomaly Detection

TL;DR

Unsupervised graph-level anomaly detection (UGAD) faces limitations from limited receptive fields and poor cross-view integration. CVTGAD introduces a Simplified Transformer with Cross-View Attention to enlarge the receptive field and bridge cross-view co-occurrence between feature and structure views through dedicated cross-view attention, enabling joint intra- and inter-graph reasoning. The approach uses a two-view graph pre-processing pipeline with GIN/GCN embeddings, a projection-residual-transformer embedding module, and an adaptive node-/graph-level scoring objective with $L = \lambda_1 L_{node} + \lambda_2 L_{graph}$ where $\lambda_1=(\sigma_{node})^{\alpha}$ and $\lambda_2=(\sigma_{graph})^{\alpha}$, achieving state-of-the-art results on 15 real-world datasets. These results demonstrate the practical significance of combining transformer-based reasoning with cross-view attention for UGAD, and the authors provide public code for reproducibility.

Abstract

Unsupervised graph-level anomaly detection (UGAD) has received remarkable performance in various critical disciplines, such as chemistry analysis and bioinformatics. Existing UGAD paradigms often adopt data augmentation techniques to construct multiple views, and then employ different strategies to obtain representations from different views for jointly conducting UGAD. However, most previous works only considered the relationship between nodes/graphs from a limited receptive field, resulting in some key structure patterns and feature information being neglected. In addition, most existing methods consider different views separately in a parallel manner, which is not able to explore the inter-relationship across different views directly. Thus, a method with a larger receptive field that can explore the inter-relationship across different views directly is in need. In this paper, we propose a novel Simplified Transformer with Cross-View Attention for Unsupervised Graph-level Anomaly Detection, namely, CVTGAD. To increase the receptive field, we construct a simplified transformer-based module, exploiting the relationship between nodes/graphs from both intra-graph and inter-graph perspectives. Furthermore, we design a cross-view attention mechanism to directly exploit the view co-occurrence between different views, bridging the inter-view gap at node level and graph level. To the best of our knowledge, this is the first work to apply transformer and cross attention to UGAD, which realizes graph neural network and transformer working collaboratively. Extensive experiments on 15 real-world datasets of 3 fields demonstrate the superiority of CVTGAD on the UGAD task. The code is available at \url{https://github.com/jindongli-Ai/CVTGAD}.

Figures (6)

• Figure 1: Toy examples to show two major issues: (a) if obtaining the embedding of the current node or graph with a limited receptive field, some nodes or patterns that carry key information would be ignored (e.g., grey in figure); (b) if ignoring the view co-occurrence across different views, some anomalous graphs would not be distinguished accurately.
• Figure 2: (a) The overview of CVTGAD. The proposed method contains three modules: graph pre-processing module, simplified transformer-based embedding module, and adaptive anomaly scoring module. (b) Specially, we extend the receptive field through a simplified transformer structure and design a cross-view attention mechanism by crossing the matrix $K$.
• Figure 3: The comparison of ablating different key components in terms of AUC.
• Figure 4: The comparison of crossing different matrices in cross-view attention mechanism in terms of AUC.
• Figure 5: The comparison of different numbers of layers for projection network and residual network in terms of AUC.