CyberGFM: Graph Foundation Models for Lateral Movement Detection in Enterprise Networks

TL;DR

CyberGFM tackles lateral movement detection by modeling enterprise network activity as a temporal graph and leveraging graph foundation models pretrained on benign random-walk sequences. It extends the DeepWalk idea by using transformer-based pretraining with scheduled masked token prediction, where tokens encode nodes and edge features, enabling rich contextual representations without prohibitive memory costs. After pretraining, CyberGFM is fine-tuned for anomaly detection via link-prediction-based or classification-based methods, achieving state-of-the-art unsupervised performance with notable improvements on three datasets. While offering substantial gains and GPU-accelerated efficiency, the approach is non-inductive and incurs inference overhead, pointing to future work in inductive extensions and scalable walk-based inference. Overall, CyberGFM demonstrates a powerful fusion of random-walk efficiency and transformer-based representation learning for scalable, precise network anomaly detection.

Abstract

Representing networks as a graph and training a link prediction model using benign connections is an effective method of anomaly-based intrusion detection. Existing works using this technique have shown great success using temporal graph neural networks and skip-gram-based approaches on random walks. However, random walk-based approaches are unable to incorporate rich edge data, while the GNN-based approaches require large amounts of memory to train. In this work, we propose extending the original insight from random walk-based skip-grams--that random walks through a graph are analogous to sentences in a corpus--to the more modern transformer-based foundation models. Using language models that take advantage of GPU optimizations, we can quickly train a graph foundation model to predict missing tokens in random walks through a network of computers. The graph foundation model is then finetuned for link prediction and used as a network anomaly detector. This new approach allows us to combine the efficiency of random walk-based methods and the rich semantic representation of deep learning methods. This system, which we call CyberGFM, achieved state-of-the-art results on three widely used network anomaly detection datasets, delivering a up to 2$\times$ improvement in average precision. We found that CyberGFM outperforms all prior works in unsupervised link prediction for network anomaly detection, using the same number of parameters, and with equal or better efficiency than the previous best approaches.

Figures (11)

• Figure 1: (a) The traditional process to pretrain LLMs samples sentences from a large corpus, masks out tokens, then optimizes the model based on its ability to predict masked tokens in the output sequence. (b) Our proposed method for GFM pretraining follows this same paradigm, using random walks through the graph as "sentences" where tokens represent node IDs and edge features.
• Figure 2: Fine-tuning options
• Figure 3: An example of going from raw log files to a CSR representation, then extracting random walks. For the standard random walk shown in the middle, the area in the dotted region ignores the temporal constraint.
• Figure 4: Change in AP score before and after fine-tuning the CyberGFM models
• Figure 5: Effect of pretraining before the finetuning stage