PrivTune: Efficient and Privacy-Preserving Fine-Tuning of Large Language Models via Device-Cloud Collaboration

TL;DR

The paper tackles privacy risks in LMaaS fine-tuning by enabling device–cloud Split Learning with on-device embedding perturbations guided by a defense-utility optimization. It introduces PrivTune, which uses a token-importance-aware $d_\chi$-Privacy mechanism and reformulates the defense problem into scalable, tractable steps (OPT-1/OPT-2 to OPT-3) with PGD, leveraging TF-IDF-like and entropy-weighted cues for token importance. The approach achieves strong protection against embedding inversion and attribute inference attacks while maintaining high task utility and low overhead, outperforming state-of-the-art baselines across five datasets and multiple model sizes. These results suggest PrivTune as a practical method for privacy-preserving personalized LM services in edge-enabled scenarios.

Abstract

With the rise of large language models, service providers offer language models as a service, enabling users to fine-tune customized models via uploaded private datasets. However, this raises concerns about sensitive data leakage. Prior methods, relying on differential privacy within device-cloud collaboration frameworks, struggle to balance privacy and utility, exposing users to inference attacks or degrading fine-tuning performance. To address this, we propose PrivTune, an efficient and privacy-preserving fine-tuning framework via Split Learning (SL). The key idea of PrivTune is to inject crafted noise into token representations from the SL bottom model, making each token resemble the $n$-hop indirect neighbors. PrivTune formulates this as an optimization problem to compute the optimal noise vector, aligning with defense-utility goals. On this basis, it then adjusts the parameters (i.e., mean) of the $d_χ$-Privacy noise distribution to align with the optimization direction and scales the noise according to token importance to minimize distortion. Experiments on five datasets (covering both classification and generation tasks) against three embedding inversion and three attribute inference attacks show that, using RoBERTa on the Stanford Sentiment Treebank dataset, PrivTune reduces the attack success rate to 10% with only a 3.33% drop in utility performance, outperforming state-of-the-art baselines.

Figures (5)

• Figure 1: Overview of the PrivTune framework, where the cloud LLM is split into a bottom model and a top model.
• Figure 2: Privacy-utility performance of RoBERTa. Note that the parts inside and outside "()" represent ASR under EIAs and AIAs respectively, and the same applies below.
• Figure 3: Privacy-utility performance of different methods on Mistral-7B.
• Figure 4: Privacy-utility performance of different methods on Llama-3.
• Figure 5: Privacy-utility performance of ScaleOT and FedBiOT.

Theorems & Definitions (1)

• Definition 1