Sample-Efficient Differentially Private Fine-Tuning via Gradient Matrix Denoising

TL;DR

The paper tackles the challenge of sample inefficiency in differentially private fine-tuning of large language models by introducing a gradient denoising post-processing step rooted in random matrix theory. By exploiting the inherent low-rank structure of gradient matrices and BBP/Marchenko-Pastur-type phase-transition insights, the method denoises noisy gradients after clipping to better align them with the clipped gradient signal. The authors derive a finite-dimensional optimal denoising procedure that operates per layer with a thresholding rule and a norm-correction mechanism to preserve gradient magnitude. Empirical results on RoBERTa fine-tuning for GLUE demonstrate faster convergence (fewer steps to target accuracy) and competitive final performance under DP-SGD, highlighting practical gains in training time without sacrificing privacy guarantees.

Abstract

We address the challenge of sample efficiency in differentially private fine-tuning of large language models (LLMs) using DP-SGD. While DP-SGD provides strong privacy guarantees, the added noise significantly increases the entropy of gradient matrices, disrupting their low-rank structure and slowing optimization. We propose a post-processing algorithm that leverages random matrix theory to denoise gradients, restore low-rank structure, and improve alignment with the original signal. Applied to DP-SGD fine-tuning of RoBERTa on GLUE tasks, our method improves sample efficiency compared to state-of-the-art approaches, substantially reducing training time when optimal performance is not required. This work demonstrates that matrix recovery techniques can enhance the utility of private language model training without compromising privacy guarantees.

Figures (4)

• Figure 1: Sorted singular values of the gradient matrix for a RoBERTa layer, before and after adding DP-SGD noise. When the signal singular values are smaller than the red line, the singular values of the noisy matrix are indistinguishable from pure noise.
• Figure 2: Sorted singular values of the gradient matrix for a RoBERTa layer, before and after adding DP-SGD noise. When some signal singular values exceed the red line, the largest singular values of the noisy matrix deviate from the bulk.
• Figure 3: Scatter plot of layer improvement vs $\frac{\lambda_1}{\sigma(\sqrt{n} + \sqrt{m})}$ for different layer dimensionality. The vertical yellow line shows the threshold $\kappa$ we used in our experiments. We want the yellow line in a position to have lots of points on top right side, and few points on the bottom right side (and preferably few on top left side).
• Figure 4: Improvement in cosine similarity between denoised and noisy gradients with respect to clipped gradients over training steps for different datasets. The positive values indicate that the denoising method consistently enhances the alignment between the noisy and clipped gradients throughout the training process.

Theorems & Definitions (2)

• Definition 1
• Definition 2