MGIC: A Multi-Label Gradient Inversion Attack based on Canny Edge Detection on Federated Learning

TL;DR

This work addresses privacy leakage in federated learning by enhancing gradient inversion (GI) attacks. MGIC introduces a multi-label GI approach that uses New Convolutional Blocks (NCB) to extract multiple labels from gradients and applies canny edge detection as a regularization cue, guided by a cosine-similarity objective: $\underset{\hat{x},\hat{y}}{\arg\min} 1 - \cos(\nabla W',\nabla W) + \mathscr{R}_{reg}$ with $\mathscr{R}_{reg} = \alpha_{TV}\mathscr{R}_{TV} + \alpha_{L2}\mathscr{R}_{L2} + \alpha_{CA}\mathscr{R}_{CA}$ and $\mathscr{R}_{CA} = \| CA_g - CA_t \|^2$. By deriving multi-labels from gradients and guiding reconstruction with edge cues, MGIC achieves higher image fidelity (PSNR/SSIM) and requires significantly less time than prior GI methods (e.g., ImageNet time ~1.38–2.76 hours vs ~8.5 hours for GGI). Experiments on ImageNet and nus-wide show MGIC reduces semantic errors and subject repetition, delivering improved reconstructions with about 20% of the previous time budget. The results underscore substantial privacy risks in FL and motivate the development of defense mechanisms against GI attacks in distributed learning systems.

Abstract

As a new distributed computing framework that can protect data privacy, federated learning (FL) has attracted more and more attention in recent years. It receives gradients from users to train the global model and releases the trained global model to working users. Nonetheless, the gradient inversion (GI) attack reflects the risk of privacy leakage in federated learning. Attackers only need to use gradients through hundreds of thousands of simple iterations to obtain relatively accurate private data stored on users' local devices. For this, some works propose simple but effective strategies to obtain user data under a single-label dataset. However, these strategies induce a satisfactory visual effect of the inversion image at the expense of higher time costs. Due to the semantic limitation of a single label, the image obtained by gradient inversion may have semantic errors. We present a novel gradient inversion strategy based on canny edge detection (MGIC) in both the multi-label and single-label datasets. To reduce semantic errors caused by a single label, we add new convolution layers' blocks in the trained model to obtain the image's multi-label. Through multi-label representation, serious semantic errors in inversion images are reduced. Then, we analyze the impact of parameters on the difficulty of input image reconstruction and discuss how image multi-subjects affect the inversion performance. Our proposed strategy has better visual inversion image results than the most widely used ones, saving more than 78% of time costs in the ImageNet dataset.

Figures (4)

• Figure 1: The processes of gradient inversion. In the typical training model, attackers will steal the gradients during the regular training process (green arrows). Attackers could build a similar image with the ground truth image by the model's backpropagation (blue arrows). Attackers will add regularization items (yellow boxes) into the loss function to acccelerate the object function coverage speed and retain the reconstructed image more naturally.
• Figure 2: Identify multi-label classification on nus-wide and ImageNet datasets. The ResNet-101 model is changed without the last AvgPooling layer and FC layer. After the image propagation through the new ResNet101 model, the attacker gets the image gradients. The NCB input is gradients multiplied by a large constant. The NCB on the nus-wide (blue blocks) and in ImageNet (green blocks) is composed as the figure shows. The size of these new layers is below layers, respectively. Finally, we get each label probability of the input data. The label's probability larger than the artificially set threshold could be deemed as the input image multi-label.
• Figure 3: Reconstructed images in the ImageNet dataset. Both GGI and MGIC are in the seeting of $restart = 1$ and $max~iteration = 20K$. The leftmost column is the ground truth image.
• Figure 4: Reconstructed images in the nus-wide dataset. Both GGI and MGIC are in the seeting of $restart = 1$ and $max~iteration = 20K$. The leftmost column is the ground truth image.