RAGFormer: Learning Semantic Attributes and Topological Structure for Fraud Detection

TL;DR

RAGFormer tackles fraud detection on multi-relational graphs by jointly learning semantic node attributes with a Transformer-based semantic encoder and topology-aware embeddings with a Relation-Aware GNN. The two representations are fused through an attentive fusion module to produce a holistic node embedding used for fraud prediction, addressing the limitations of using either perspective alone. Empirical results on YelpChi, Amazon, and Pay show state-of-the-art performance and strong data-efficiency, with notable AP gains on YelpChi and consistent improvements on financial fraud detection. The approach offers practical benefits for real-world fraud systems, albeit with increased computational cost due to the dual-model architecture.

Abstract

Fraud detection remains a challenging task due to the complex and deceptive nature of fraudulent activities. Current approaches primarily concentrate on learning only one perspective of the graph: either the topological structure of the graph or the attributes of individual nodes. However, we conduct empirical studies to reveal that these two types of features, while nearly orthogonal, are each independently effective. As a result, previous methods can not fully capture the comprehensive characteristics of the fraud graph. To address this dilemma, we present a novel framework called Relation-Aware GNN with transFormer~(RAGFormer) which simultaneously embeds both semantic and topological features into a target node. The simple yet effective network consists of a semantic encoder, a topology encoder, and an attention fusion module. The semantic encoder utilizes Transformer to learn semantic features and node interactions across different relations. We introduce Relation-Aware GNN as the topology encoder to learn topological features and node interactions within each relation. These two complementary features are interleaved through an attention fusion module to support prediction by both orthogonal features. Extensive experiments on two popular public datasets demonstrate that RAGFormer achieves state-of-the-art performance. The significant improvement of RAGFormer in an industrial credit card fraud detection dataset further validates the applicability of our method in real-world business scenarios.

Figures (7)

• Figure 1: Similarity between features learned by our semantic encoder(Transformer) and topology encoder(GNN). A value of 0, indicated by the red line, signifies that the features are orthogonal.
• Figure 2: The pipeline of our RAGFormer method that learns both semantic features and topological features. The complete multi-relation graph $\mathcal{G}$ is input into the semantic encoder. The topology encoder contains $R$ independent GNNs. We extract the relation-specific subgraphs $\mathcal{G}_{1},...,\mathcal{G}_{R}$ as the input of the corresponding GNN. The semantic features $X_{sem}$ and the topological features $X_{gcn}$ are then input into the feature fusion module to get the final embeddings $X_{fused}$. A classification head, with the Multi-Layer Perceptron (MLP) architecture, takes $X_{fused}$ as the input and outputs the classification probabilities $\hat{\mathcal{Y}}$. We set the relation number $R$ to 2 within the graph for simplicity.
• Figure 3: The comparison of method performance of our RAGFormer, GAGA and CAREGNN on YelpChi with variable training ratios ranging from 10% to 40%. We evaluate all methods under AUC, AP, and F1-macro ((a), (b), (c)).
• Figure 4: Ablation study on how the semantic encoder and topology encoder contribute to the proposed RAGFormer.
• Figure 4: Hyper-parameter sensitivity analysis with respect to (a) the number of layers in the semantic encoder (b) the number of layers in the topology encoder (c) the embedding dimension of each relation and (d) the number of hops. All experiments are conducted on YelpChi.