Large Language Models Merging for Enhancing the Link Stealing Attack on Graph Neural Networks

TL;DR

The paper addresses privacy risks in graph neural networks by enabling cross-dataset link stealing through fine-tuned large language models and a novel model-merging pipeline. By training attacker-specific LLMs on private datasets and merging their delta parameters via Drop, Elect, and Merge with data-sharing avoidance, the approach yields a universal attack model with strong generalization, including to unseen datasets. Theoretical analysis shows LLMs enable cross-dataset attacks and that the merging procedure preserves attack efficacy, while extensive experiments across four datasets and multiple GNN/LLM architectures demonstrate superior performance and robustness, including notable out-of-domain results. This work highlights a practical, scalable threat model for GNN privacy attacks and suggests critical considerations for defending against multi-source, cross-domain adversaries in graph-based systems.

Abstract

Graph Neural Networks (GNNs), specifically designed to process the graph data, have achieved remarkable success in various applications. Link stealing attacks on graph data pose a significant privacy threat, as attackers aim to extract sensitive relationships between nodes (entities), potentially leading to academic misconduct, fraudulent transactions, or other malicious activities. Previous studies have primarily focused on single datasets and did not explore cross-dataset attacks, let alone attacks that leverage the combined knowledge of multiple attackers. However, we find that an attacker can combine the data knowledge of multiple attackers to create a more effective attack model, which can be referred to cross-dataset attacks. Moreover, if knowledge can be extracted with the help of Large Language Models (LLMs), the attack capability will be more significant. In this paper, we propose a novel link stealing attack method that takes advantage of cross-dataset and Large Language Models (LLMs). The LLM is applied to process datasets with different data structures in cross-dataset attacks. Each attacker fine-tunes the LLM on their specific dataset to generate a tailored attack model. We then introduce a novel model merging method to integrate the parameters of these attacker-specific models effectively. The result is a merged attack model with superior generalization capabilities, enabling effective attacks not only on the attackers' datasets but also on previously unseen (out-of-domain) datasets. We conducted extensive experiments in four datasets to demonstrate the effectiveness of our method. Additional experiments with three different GNN and LLM architectures further illustrate the generality of our approach.

Figures (11)

• Figure 1: Overview of general link stealing attacks. The upper section depicts the service provider offering the target model for user access, while the lower section illustrates attackers executing the attack process.
• Figure 2: Overview of the proposed link stealing attack method. The upper section illustrates the service provider offering the target model for user access. The lower section details the attackers’ process of creating attack models using our proposed method. In Attacker Step 1, Attackers introduce LLMs and leverage their respective node features to fine-tune and develop multiple LLM-based link stealing attack models. In Attacker Step 2, the attackers, through model merging techniques, combine the strengths of these individual models to create a more robust merged LLM model for executing link stealing attacks.
• Figure 3: Prompts designed for LLM-based link stealing attacks.
• Figure 4: Schematic diagram of mapping weights to be inversely proportional to dropout probability.
• Figure 5: Illustrative example of the weight calculation process for $\lambda$.