Memorization in NLP Fine-tuning Methods

TL;DR

This work probes memorization risks during fine-tuning of GPT-2 across three methods—full fine-tuning, head-only tuning, and adapters—using membership inference recall and exposure as metrics on multiple datasets. It reveals that head-tuning exhibits substantially higher leakage, while full fine-tuning and adapters achieve more favorable privacy-utility trade-offs, forming a Pareto frontier. Through ablations on parameter count, location, and tying, the study shows that where and how parameters are trained strongly influences memorization, not just how many are trained. The findings inform privacy-aware fine-tuning practices, suggesting that adapters with substantial bottleneck reductions or full fine-tuning are preferable when privacy risks are a concern.

Abstract

Large language models are shown to present privacy risks through memorization of training data, and several recent works have studied such risks for the pre-training phase. Little attention, however, has been given to the fine-tuning phase and it is not well understood how different fine-tuning methods (such as fine-tuning the full model, the model head, and adapter) compare in terms of memorization risk. This presents increasing concern as the "pre-train and fine-tune" paradigm proliferates. In this paper, we empirically study memorization of fine-tuning methods using membership inference and extraction attacks, and show that their susceptibility to attacks is very different. We observe that fine-tuning the head of the model has the highest susceptibility to attacks, whereas fine-tuning smaller adapters appears to be less vulnerable to known extraction attacks.

Figures (11)

• Figure 1: Each point in the graph shows the given metric values at the end of each training epoch. The rightmost lower points show the beginning, and as we move to left and upwards training progresses. We identify three separate phases within the learning process, distinguished by their memorization and generalization trends.
• Figure 2: Pareto frontier for utility (validation PPL) Vs. privacy (MIA recall). Each dot shows different checkpoints, and the colors show different fine-tuning methods. We desire models that have low PPL and low attack recall.
• Figure 3: Ablating how the location and number of trainable parameters effects memorization on the Penn Treebank dataset. Each dot shows different checkpoints, and the colors show different fine-tuning methods. We desire models that have low PPL and low attack recall.
• Figure 4: Ablating how the untying of the trainable parameters effects memorization on the Penn Treebank dataset. Each dot shows different checkpoints, and the colors show different fine-tuning methods. We desire models that have low PPL and low attack recall.
• Figure 5: Ablating how training the model from scratch affects the privacy-utility trade-off, compared to fine-tuning a pre-trained model, on the Wikipedia dataset. Each dot shows different checkpoints, and the colors show different fine-tuning methods. We desire models that have low PPL and low attack recall.