Unveiling the Vulnerability of Private Fine-Tuning in Split-Based Frameworks for Large Language Models: A Bidirectionally Enhanced Attack

TL;DR

The paper addresses privacy risks in split-based fine-tuning of large language models by uncovering two vulnerabilities—the Not-too-far effect of relying on pre-trained weights and the autoregressive training of LLMs—that facilitate data reconstruction from smashed data and gradients. It introduces BiSR, a bidirectional attack that combines a learning-based inversion (SIP) with optimization-based refinement (BRE), augmented by a Noise-adaptive Mixture of Experts (NaMoE) to handle forward perturbations. Across diverse models, datasets, and defense mechanisms, BiSR achieves state-of-the-art reconstruction, demonstrating strong cross-dataset and cross-layer transfer and resilience to perturbations, thereby exposing serious privacy risks in MaaS split-learning for LLMs. The findings motivate the development of robust defenses and careful consideration of data and model ownership in split-based private fine-tuning frameworks, given that even semi-white-box access can enable substantial data leakage. Overall, the work provides a comprehensive methodology and empirical evidence that bidirectional reconstruction attacks threaten private fine-tuning workflows and highlights actionable directions for future defense research.

Abstract

Recent advancements in pre-trained large language models (LLMs) have significantly influenced various domains. Adapting these models for specific tasks often involves fine-tuning (FT) with private, domain-specific data. However, privacy concerns keep this data undisclosed, and the computational demands for deploying LLMs pose challenges for resource-limited data holders. This has sparked interest in split learning (SL), a Model-as-a-Service (MaaS) paradigm that divides LLMs into smaller segments for distributed training and deployment, transmitting only intermediate activations instead of raw data. SL has garnered substantial interest in both industry and academia as it aims to balance user data privacy, model ownership, and resource challenges in the private fine-tuning of LLMs. Despite its privacy claims, this paper reveals significant vulnerabilities arising from the combination of SL and LLM-FT: the Not-too-far property of fine-tuning and the auto-regressive nature of LLMs. Exploiting these vulnerabilities, we propose Bidirectional Semi-white-box Reconstruction (BiSR), the first data reconstruction attack (DRA) designed to target both the forward and backward propagation processes of SL. BiSR utilizes pre-trained weights as prior knowledge, combining a learning-based attack with a bidirectional optimization-based approach for highly effective data reconstruction. Additionally, it incorporates a Noise-adaptive Mixture of Experts (NaMoE) model to enhance reconstruction performance under perturbation. We conducted systematic experiments on various mainstream LLMs and different setups, empirically demonstrating BiSR's state-of-the-art performance. Furthermore, we thoroughly examined three representative defense mechanisms, showcasing our method's capability to reconstruct private data even in the presence of these defenses.

Figures (19)

• Figure 1: The private-label framework for fine-tuning LLaMA2-chat-7B, illustrated with a single client. The model's 32 decoder blocks are divided into three segments: Bottom (embedding layer and the first two blocks), Trunk (middle 28 blocks), and Top (final two blocks, normalization layer, and linear layer). The client hosts the Bottom and Top segments, while the server manages the Trunk.
• Figure 2: The attack framework (middle) targeting the split-- system (left). This two-stage attack starts with a learning-based approach, where the -trained attack model ( model) performs direct inversion on smashed data. The second stage builds upon the sentences initially recovered, employing both forward (using smashed data) and backward (using gradients) data for bidirectional, optimization-based enhancement. This process, detailed on the right, results in significantly more comprehensive sentence recovery.
• Figure 3: The proposed m1. During the training phase (left), the encoder mimicking the target model's Bottom segment but with pre-trained parameters is frozen, while the decoder is trained on a dataset $D_a$ similar to $D_{c_i}$, with original inputs $b \in D_{a}$ as labels. In the attack stage (right), the inversion model receives the real model's Bottom output $\tilde{x}_{\text{btm}}$, the smashed data, and directly decodes it to obtain the attack results $\hat{x}$.
• Figure 4: Demonstration of the impact of 's learning-based initialization () on optimization-based process: providing an effective starting point for achieving global optima.
• Figure 5: Performance boost brought by noise-aware training, demonstrated by attack performance (ROUGE-L F1 Score %) of inverters trained with embedding--awareness on split- systems using embedding-glsdxp with varying noise scales, evaluated on GPT2-large and PIQA datasets.