GraphLoRA: Structure-Aware Contrastive Low-Rank Adaptation for Cross-Graph Transfer Learning

TL;DR

GraphLoRA tackles cross-graph transfer learning for GNNs under feature and structural distribution shifts and scarce target labels. It combines a Structure-aware Maximum Mean Discrepancy (SMMD) for node-feature alignment, a low-rank adaptation GNN to bridge structural gaps with a graph contrastive objective, and a structure-aware regularization to exploit homophily with limited supervision. The approach is parameter-efficient, freezing the pre-trained backbone and tuning a small adapter, achieving consistent gains across eight real-world datasets and several few-shot settings, while mitigating catastrophic forgetting. Results demonstrate improved transferability and scalability, with strong empirical and theoretical support for robust cross-graph representations and practical impact for deploying GNNs across diverse graphs.

Abstract

Graph Neural Networks (GNNs) have demonstrated remarkable proficiency in handling a range of graph analytical tasks across various domains, such as e-commerce and social networks. Despite their versatility, GNNs face significant challenges in transferability, limiting their utility in real-world applications. Existing research in GNN transfer learning overlooks discrepancies in distribution among various graph datasets, facing challenges when transferring across different distributions. How to effectively adopt a well-trained GNN to new graphs with varying feature and structural distributions remains an under-explored problem. Taking inspiration from the success of Low-Rank Adaptation (LoRA) in adapting large language models to various domains, we propose GraphLoRA, an effective and parameter-efficient method for transferring well-trained GNNs to diverse graph domains. Specifically, we first propose a Structure-aware Maximum Mean Discrepancy (SMMD) to align divergent node feature distributions across source and target graphs. Moreover, we introduce low-rank adaptation by injecting a small trainable GNN alongside the pre-trained one, effectively bridging structural distribution gaps while mitigating the catastrophic forgetting. Additionally, a structure-aware regularization objective is proposed to enhance the adaptability of the pre-trained GNN to target graph with scarce supervision labels. Extensive experiments on eight real-world datasets demonstrate the effectiveness of GraphLoRA against fourteen baselines by tuning only 20% of parameters, even across disparate graph domains. The code is available at https://github.com/AllminerLab/GraphLoRA.

Figures (5)

• Figure 1: Negative transfer occurs in cross-graph adaptation, where PubMed, CiteSeer, and Cora are citation networks, whereas Photo and Computer are co-purchase networks.
• Figure 2: The framework of GraphLoRA: Fine-tuning the pre-trained GNN for the target graph. The node feature adaptation and structural knowledge transfer learning modules are designed to alleviate feature and structural discrepancies, respectively. Furthermore, the structure-aware regularization objective is crafted to enhance the adaptability of the pre-trained GNN.
• Figure 3: Experimental results across different shots.
• Figure 4: Performance across varying hyperparameter values.
• Figure 5: Visualization of node embeddings on CiteSeer.

Theorems & Definitions (6)

• Theorem 1
• Lemma 1
• Lemma 2
• Lemma 3
• Theorem 1
• Theorem 2