SPEAR++: Scaling Gradient Inversion via Sparsely-Used Dictionary Learning
Alexander Bakarsky, Dimitar I. Dimitrov, Maximilian Baader, Martin Vechev
TL;DR
This paper tackles the privacy risk in Federated Learning posed by gradient inversion attacks by scaling exact inversions for linear layers with ReLU activations. It reframes gradient leakage as a Sparsely-Used Dictionary Learning problem, replacing the prior exponential subproblem with scalable dictionary-learning-based search and a sampling-based rounding step. The authors demonstrate that SPEAR++ achieves much higher batch-size scalability (about 10x) while preserving robustness to DP noise and FedAvg aggregation, providing a practical attack with strong reconstruction performance. The work highlights a stronger privacy vulnerability in real-world FL deployments and offers a concrete methodology for attackers and a framework for evaluating defenses.
Abstract
Federated Learning has seen an increased deployment in real-world scenarios recently, as it enables the distributed training of machine learning models without explicit data sharing between individual clients. Yet, the introduction of the so-called gradient inversion attacks has fundamentally challenged its privacy-preserving properties. Unfortunately, as these attacks mostly rely on direct data optimization without any formal guarantees, the vulnerability of real-world systems remains in dispute and requires tedious testing for each new federated deployment. To overcome these issues, recently the SPEAR attack was introduced, which is based on a theoretical analysis of the gradients of linear layers with ReLU activations. While SPEAR is an important theoretical breakthrough, the attack's practicality was severely limited by its exponential runtime in the batch size b. In this work, we fill this gap by applying State-of-the-Art techniques from Sparsely-Used Dictionary Learning to make the problem of gradient inversion on linear layers with ReLU activations tractable. Our experiments demonstrate that our new attack, SPEAR++, retains all desirable properties of SPEAR, such as robustness to DP noise and FedAvg aggregation, while being applicable to 10x bigger batch sizes.