DynBERG: Dynamic BERT-based Graph neural network for financial fraud detection
Omkar Kulkarni, Rohitash Chandra
TL;DR
DynBERG tackles financial fraud detection on dynamic, directed transaction graphs by marrying Graph-BERT's transformer-based graph encoding with a GRU to capture temporal evolution. The approach uses subgraph batching to manage large, evolving graphs and introduces directed-edge normalization, a time-aware pre-training objective, and GRU-based fusion for dynamic node classification. Empirical evaluation on the Elliptic dataset shows DynBERG outperforming static and dynamic baselines before a major market disruption, with ablations confirming the temporal component's value, though post-disruption performance declines across models. The work highlights both the potential and the need for adaptive, real-time updating mechanisms to handle rapid shifts in fraud patterns in financial networks.
Abstract
Financial fraud detection is critical for maintaining the integrity of financial systems, particularly in decentralised environments such as cryptocurrency networks. Although Graph Convolutional Networks (GCNs) are widely used for financial fraud detection, graph Transformer models such as Graph-BERT are gaining prominence due to their Transformer-based architecture, which mitigates issues such as over-smoothing. Graph-BERT is designed for static graphs and primarily evaluated on citation networks with undirected edges. However, financial transaction networks are inherently dynamic, with evolving structures and directed edges representing the flow of money. To address these challenges, we introduce DynBERG, a novel architecture that integrates Graph-BERT with a Gated Recurrent Unit (GRU) layer to capture temporal evolution over multiple time steps. Additionally, we modify the underlying algorithm to support directed edges, making DynBERG well-suited for dynamic financial transaction analysis. We evaluate our model on the Elliptic dataset, which includes Bitcoin transactions, including all transactions during a major cryptocurrency market event, the Dark Market Shutdown. By assessing DynBERG's resilience before and after this event, we analyse its ability to adapt to significant market shifts that impact transaction behaviours. Our model is benchmarked against state-of-the-art dynamic graph classification approaches, such as EvolveGCN and GCN, demonstrating superior performance, outperforming EvolveGCN before the market shutdown and surpassing GCN after the event. Additionally, an ablation study highlights the critical role of incorporating a time-series deep learning component, showcasing the effectiveness of GRU in modelling the temporal dynamics of financial transactions.