Revisiting Privacy, Utility, and Efficiency Trade-offs when Fine-Tuning Large Language Models

TL;DR

This paper investigates the privacy, utility, and efficiency trade-offs during fine-tuning of large language models and introduces a token-sensitivity framework that separates privacy (sensitive-token memory) from utility (non-sensitive-token performance). It empirically compares three fine-tuning methods—full fine-tuning, differential privacy, and Low-Rank Adaptation (LoRA)—across multiple open-source LLM families and domain datasets, using new metrics that distinguish sensitive vs non-sensitive tokens. The key finding is that LoRA achieves privacy comparable to DP while delivering similar utility and far greater efficiency, challenging the conventional view that privacy and efficiency are at odds. The results suggest that privacy-aware, parameter-efficient fine-tuning can simultaneously meet privacy, utility, and efficiency goals, motivating broader cross-disciplinary exploration.

Abstract

We study the inherent trade-offs in minimizing privacy risks and maximizing utility, while maintaining high computational efficiency, when fine-tuning large language models (LLMs). A number of recent works in privacy research have attempted to mitigate privacy risks posed by memorizing fine-tuning data by using differentially private training methods (e.g., DP), albeit at a significantly higher computational cost (inefficiency). In parallel, several works in systems research have focussed on developing (parameter) efficient fine-tuning methods (e.g., LoRA), but few works, if any, investigated whether such efficient methods enhance or diminish privacy risks. In this paper, we investigate this gap and arrive at a surprising conclusion: efficient fine-tuning methods like LoRA mitigate privacy risks similar to private fine-tuning methods like DP. Our empirical finding directly contradicts prevailing wisdom that privacy and efficiency objectives are at odds during fine-tuning. Our finding is established by (a) carefully defining measures of privacy and utility that distinguish between memorizing sensitive and non-sensitive tokens in training and test datasets used in fine-tuning and (b) extensive evaluations using multiple open-source language models from Pythia, Gemma, and Llama families and different domain-specific datasets.

Figures (27)

• Figure 1: Sensitive and non-sensitive tokens have different predictability, measured as the recollection loss by pre-trained models in Figure \ref{['fig:a']} and fine-tuned models (recorded at epoch $5$) in Figure \ref{['fig:b']}. The distinction motivates us to quantify privacy using sensitive token loss on training data (higher is better) and utility as non-sensitive token loss on test data (lower is better).
• Figure 2: Our measures offer a more precise assessment of privacy and utility when fine-tuning LLMs by distinguishing between sensitive and non-sensitive tokens, revealing higher privacy (higher loss) for sensitive tokens and better utility (lower loss) for non-sensitive tokens compared to traditional measures that overlook this sensitivity-based distinction.
• Figure 3: GPT-4 shows higher annotation accuracy, with 75% of participants rating its annotations to be accurate while Presidio annotations were mostly mixed or under-annotated.
• Figure 4: Memorized sequences are predominantly sourced from GitHub and ArXiv, despite these sections being mid-range in the original Pile dataset, suggesting that memorized content is largely non-sensitive and may pose a lower privacy risk than previously assumed.
• Figure 5: GPT-4 achieves an average accuracy of 78% in predicting the source of memorized strings across Pile dataset sections, reinforcing the reliability of GPT-4 and supporting our position that privacy concerns in prior work are overestimated without distinguishing token sensitivity.