Leak and Learn: An Attacker's Cookbook to Train Using Leaked Data from Federated Learning

TL;DR

This work reframes data reconstruction attacks in federated learning as a question of downstream usefulness, showing that leaked data can train models that outperform standard FedAvg and even approach centralized performance under favorable conditions. It systematically evaluates gradient inversion and linear layer leakage across MNIST, CIFAR-10, and Tiny ImageNet, revealing that GI quality declines with batch size and that LLL can leak substantial data but struggles with label matching; SSL and initialization from FedAvg can mitigate some of these issues. The study highlights a nuanced privacy-utility tradeoff: leaked data can be repurposed to train competitive models, but practical deployment faces challenges in labeling, data quality, and scalability, underscoring the need for defenses and further research. Overall, the results demonstrate both the potential risks of leakage in FL and the complexities involved in translating leaked data into effective downstream models.

Abstract

Federated learning is a decentralized learning paradigm introduced to preserve privacy of client data. Despite this, prior work has shown that an attacker at the server can still reconstruct the private training data using only the client updates. These attacks are known as data reconstruction attacks and fall into two major categories: gradient inversion (GI) and linear layer leakage attacks (LLL). However, despite demonstrating the effectiveness of these attacks in breaching privacy, prior work has not investigated the usefulness of the reconstructed data for downstream tasks. In this work, we explore data reconstruction attacks through the lens of training and improving models with leaked data. We demonstrate the effectiveness of both GI and LLL attacks in maliciously training models using the leaked data more accurately than a benign federated learning strategy. Counter-intuitively, this bump in training quality can occur despite limited reconstruction quality or a small total number of leaked images. Finally, we show the limitations of these attacks for downstream training, individually for GI attacks and for LLL attacks.

Figures (9)

• Figure 1: Training using leaked data.
• Figure 2: Training models on (a) CIFAR-10, (b) MNIST, and (c) Tiny ImageNet with leaked data compared to centralized and federated learning training. Both linear layer leakage and gradient inversion achieve higher accuracy than the federated learning (FedAvg) baseline for CIFAR-10 in all cases. For MNIST, LLL (LOKI) nearly reaches centralized accuracy while GI performs slightly worse than FL. Top-1 validation accuracy used when training models on Tiny ImageNet. Here, LLL performs better with a FC layer size factor of 2 or higher.
• Figure 3: Semi-supervised learning using CoMatch on CIFAR-10 with a WideResNet with (a) a varying FC size and leakage rate (LR) for LOKI and the known labels fixed at 40 and (b) a fixed number of leaked images and 20, 40, and 250 known labels.
• Figure 4: FL model trained with 50 clients as start point for training with leaked LLL data. Highlighted area indicates improvement above the FL model and training on the leaked data alone.
• Figure 5: Reconstructions on CIFAR-10 from Inverting Gradients batch size 8. Ground truth images are on top and reconstructions are on the bottom. All labels in the batch are different and reconstructed images are high quality.