The Federation Strikes Back: A Survey of Federated Learning Privacy Attacks, Defenses, Applications, and Policy Landscape

TL;DR

This survey tackles the privacy challenges of Federated Learning by cataloging attacks (data reconstruction, membership inference, property inference, and model extraction) and defenses (differential privacy, secure aggregation, and homomorphic encryption) across FL variants (cross-device, cross-silo, horizontal, vertical, hierarchical, and FL-derived forms). It grounds technical discussions in concrete application domains (healthcare, finance, IoT/edge) and examines the evolving policy landscape (GDPR, US privacy laws, AI-related acts) shaping deployment. A central contribution is the synthesis of attack-defense-tPolicy triad with real-world deployment insights and open questions, highlighting the need for certifiable protections, realistic utility-privacy tradeoffs, and robust handling of non-IID data and heterogeneous devices. The work emphasizes that, despite FL’s privacy-by-design promise, sophisticated attacks persist, motivating multi-layer defenses and regulatory alignment to enable practical, privacy-preserving FL in safety- and privacy-critical applications.

Abstract

Deep learning has shown incredible potential across a wide array of tasks, and accompanied by this growth has been an insatiable appetite for data. However, a large amount of data needed for enabling deep learning is stored on personal devices, and recent concerns on privacy have further highlighted challenges for accessing such data. As a result, federated learning (FL) has emerged as an important privacy-preserving technology that enables collaborative training of machine learning models without the need to send the raw, potentially sensitive, data to a central server. However, the fundamental premise that sending model updates to a server is privacy-preserving only holds if the updates cannot be "reverse engineered" to infer information about the private training data. It has been shown under a wide variety of settings that this privacy premise does not hold. In this survey paper, we provide a comprehensive literature review of the different privacy attacks and defense methods in FL. We identify the current limitations of these attacks and highlight the settings in which the privacy of an FL client can be broken. We further dissect some of the successful industry applications of FL and draw lessons for future successful adoption. We survey the emerging landscape of privacy regulation for FL and conclude with future directions for taking FL toward the cherished goal of generating accurate models while preserving the privacy of the data from its participants.

Figures (6)

• Figure 1: Images reconstructed by Inverting Gradients geiping2020inverting, an optimization attack, on a batch of 8 and 16 CIFAR-10 images in FedSGD. Ground truth images (a, c) are shown on the left, and reconstructions (b, d) are shown on the right. Images are visually recognizable for batch size 8, but become less recognizable at batch size 16.
• Figure 2: Images reconstructed using LOKI zhao2023loki, a linear layer leakage attack, on a batch of 64 CIFAR-10 images. FedAvg with 8 local iterations of mini-batch size 8 used. Out of 64 images, 55 are reconstructed successfully with high quality.
• Figure 3: Using the weight gradient $\frac{\delta L}{\delta W}$ and bias gradient $\frac{\delta L}{\delta B}$ of a fully connected layer to reconstruct the inputs. Neuron $i$ is only activated by a single image, while $j$ is activated by two. As a result, the reconstruction of neuron $i$ is correct while $j$ is a combination of images.
• Figure 4: Membership inference attack workflow from zari2021efficient. The attacker first collects information (e.g. parameter updates) from clients, then uses auxiliary dataset to train shadow models which can mimic the whole federated learning procedure, so that one attack model can be trained using the information collected from shadow models. Lastly, the attacker feeds the information collected from real clients into the attack model to perform membership inference attacks.
• Figure 5: Representative example of orthogonality of gradients of distinct instances (at epoch 250, AlexNet, CIFAR-100 dataset) from li2022effective: (a) Distribution of cosine similarity (cosim) between gradients of two distinct member instances. (b) Distribution of cosine similarity between gradients of one member instance and one non-member instance. (c) Distribution of cosine similarity between gradients of two distinct non-member instances.