Exploring User-level Gradient Inversion with a Diffusion Prior

TL;DR

A novel gradient inversion attack that applies a denoising diffusion model as a strong image prior in order to enhance recovery in the large batch setting in order to recover realistic facial images along with private user attributes.

Abstract

We explore user-level gradient inversion as a new attack surface in distributed learning. We first investigate existing attacks on their ability to make inferences about private information beyond training data reconstruction. Motivated by the low reconstruction quality of existing methods, we propose a novel gradient inversion attack that applies a denoising diffusion model as a strong image prior in order to enhance recovery in the large batch setting. Unlike traditional attacks, which aim to reconstruct individual samples and suffer at large batch and image sizes, our approach instead aims to recover a representative image that captures the sensitive shared semantic information corresponding to the underlying user. Our experiments with face images demonstrate the ability of our methods to recover realistic facial images along with private user attributes.

Figures (8)

• Figure 1: Illustration of user-level gradient inversion with diffusion prior. Top: user's private batch of $30$ images. The measured pairwise facial similarity in the private batch ranges from 0.5217 to 0.9579. Bottom: reconstructed image from gradients using the proposed method. The average facial similarity to the original batch is 0.5565.
• Figure 2: Visual comparison of reconstruction results from a batch of $30$ images.
• Figure 3: Visual comparison of reconstruction results (full batch).
• Figure 4: Example of reconstructing from a batch of 100 images. The measured pairwise facial similarity in the private batch ranges from 0.4371 to 0.9528. The average facial similarity of the reconstructed image to the original batch is 0.7080.
• Figure 5: Time schedule and sliding window for optimization.