Graph Attention-based Adaptive Transfer Learning for Link Prediction

TL;DR

GAATNet tackles link prediction on large-scale sparse graphs by uniting pretraining on large source graphs with a parameter-efficient fine-tuning stage on target graphs. It introduces diffusion-based data augmentation to enrich embeddings, and a distant-neighbor bias in self-attention to capture global topology, complemented by a lightweight self-adapter in the fine-tuning network. The model optimizes with a joint loss comprising $L_{link}$ and $L_{con}$ and achieves strong empirical gains across seven public datasets while reducing training costs through a reduced parameter footprint $O(d' q)$ during fine-tuning. These design choices enable robust cross-graph transfer, improved generalization, and scalable LP performance in sparse settings, with potential extensions to dynamic and heterogeneous graphs. Overall, GAATNet provides a practical, scalable framework for transferring rich graph representations across datasets while maintaining efficiency and accuracy in link prediction.

Abstract

Graph neural networks (GNNs) have brought revolutionary advancements to the field of link prediction (LP), providing powerful tools for mining potential relationships in graphs. However, existing methods face challenges when dealing with large-scale sparse graphs and the need for a high degree of alignment between different datasets in transfer learning. Besides, although self-supervised methods have achieved remarkable success in many graph tasks, prior research has overlooked the potential of transfer learning to generalize across different graph datasets. To address these limitations, we propose a novel Graph Attention Adaptive Transfer Network (GAATNet). It combines the advantages of pre-training and fine-tuning to capture global node embedding information across datasets of different scales, ensuring efficient knowledge transfer and improved LP performance. To enhance the model's generalization ability and accelerate training, we design two key strategies: 1) Incorporate distant neighbor embeddings as biases in the self-attention module to capture global features. 2) Introduce a lightweight self-adapter module during fine-tuning to improve training efficiency. Comprehensive experiments on seven public datasets demonstrate that GAATNet achieves state-of-the-art performance in LP tasks. This study provides a general and scalable solution for LP tasks to effectively integrate GNNs with transfer learning. The source code and datasets are publicly available at https://github.com/DSI-Lab1/GAATNet

Figures (9)

• Figure 1: An overview of our transfer learning framework. Positive samples represent observed edges (label = 1), while negative samples represent unobserved edges (label = 0). In the feature space, a nonlinear method is used to compute $y_{\textit{pred}}$. If $y_{\textit{pred}} > \theta$, it indicates a potential edge formation. Here $\theta$ is a given threshold, typically set to 0.5.
• Figure 2: Overview of the GAATNet framework. The pre-training network (left) incorporates distant neighbor embeddings as attention biases. The fine-tuning network (right) uses a lightweight self-adapter module to boost efficiency and reduce trainable parameters. Both parts are trained under a joint objective combining contrastive loss and link prediction loss.
• Figure 3: The process of aggregating node features from neighboring information in GAT.
• Figure 4: Effectiveness of the pre-training. $\text{GAATNet}_s$ represents a model trained from scratch without loading pretrained model parameters. $\text{GAATNet}_t$ refers to a model that loads Transformer-based pretrained parameters and fine-tunes based on it.
• Figure 5: Effectiveness of the fine-tuning.