TGTOD: A Global Temporal Graph Transformer for Outlier Detection at Scale
Kay Liu, Jiahao Ding, MohamadAli Torkamani, Philip S. Yu
TL;DR
TGTOD introduces a global spatiotemporal attention framework for temporal graphs to address outlier detection at scale. By partitioning the graph into spatiotemporal patches and processing them with a hierarchical Transformer (PFormer, CFormer, TFormer) plus a GNN-augmented Patch Transformer, it achieves end-to-end node-level detection with improved generalization over link-prediction pretraining. The approach significantly reduces attention complexity and demonstrates superior accuracy (e.g., AP on Elliptic) and efficiency (up to 4x faster training) across three real-world datasets. This work establishes a scalable baseline for temporal graph outlier detection and motivates extensions to broader spatiotemporal learning tasks.
Abstract
While Transformers have revolutionized machine learning on various data, existing Transformers for temporal graphs face limitations in (1) restricted receptive fields, (2) overhead of subgraph extraction, and (3) suboptimal generalization capability beyond link prediction. In this paper, we rethink temporal graph Transformers and propose TGTOD, a novel end-to-end Temporal Graph Transformer for Outlier Detection. TGTOD employs global attention to model both structural and temporal dependencies within temporal graphs. To tackle scalability, our approach divides large temporal graphs into spatiotemporal patches, which are then processed by a hierarchical Transformer architecture comprising Patch Transformer, Cluster Transformer, and Temporal Transformer. We evaluate TGTOD on three public datasets under two settings, comparing with a wide range of baselines. Our experimental results demonstrate the effectiveness of TGTOD, achieving AP improvement of 61% on Elliptic. Furthermore, our efficiency evaluation shows that TGTOD reduces training time by 44x compared to existing Transformers for temporal graphs. To foster reproducibility, we make our implementation publicly available at https://github.com/kayzliu/tgtod.