Hiding in Plain Sight: Disguising Data Stealing Attacks in Federated Learning

TL;DR

The paper addresses privacy vulnerabilities in federated learning posed by malicious servers, focusing on client-side detectability. It demonstrates that existing malicious-server attacks are detectable under principled checks and introduces SEER, a secret-decoder-based framework that embeds data disaggregation in a hidden gradient space to reconstruct client data, even with large batches and secure aggregation. SEER is trained end-to-end with a shared model and a secret decoder/reconstructor, and its efficacy is validated through extensive experiments on CIFAR-10/100 and ImageNet, showing high reconstruction rates and robustness to distribution shifts. The work argues for principled, client-side defenses and provides a foundation for evaluating and improving FL privacy protections in real-world deployments.

Abstract

Malicious server (MS) attacks have enabled the scaling of data stealing in federated learning to large batch sizes and secure aggregation, settings previously considered private. However, many concerns regarding the client-side detectability of MS attacks were raised, questioning their practicality. In this work, for the first time, we thoroughly study client-side detectability. We first demonstrate that all prior MS attacks are detectable by principled checks, and formulate a necessary set of requirements that a practical MS attack must satisfy. Next, we propose SEER, a novel attack framework that satisfies these requirements. The key insight of SEER is the use of a secret decoder, jointly trained with the shared model. We show that SEER can steal user data from gradients of realistic networks, even for large batch sizes of up to 512 and under secure aggregation. Our work is a promising step towards assessing the true vulnerability of federated learning in real-world settings.

Figures (14)

• Figure 1: D-SNR (\ref{['sec:detection']}) of real (model, data batch) pairs. High values indicate vulnerability to data leakage, which can manifest even in non-malicious models (). Example disaggregation attacks () are easily detectable as they can successfully reconstruct data () only when DSNR is unusually high (note the log scale), and fail otherwise (). Our method, SEER (, \ref{['sec:attack']}), successfully reconstructs an example even when D-SNR is low (), and is thus hard to detect in the original gradient space.
• Figure 2: Overview of SEER. A client propagates a batch ${{\bm{X}}}$ (of which one image satisfies the property $\mathcal{P}$ only known to the server) through the shared network $f$ with malicious weights ${\bm \theta}_f$, and returns the aggregated gradient ${\bm{g}}$, hoping that the aggregation protects individual images. The server steals the image satisfying $\mathcal{P}$ by applying a secret disaggregator $d$ to remove the impact of other images in a hidden space, followed by a secret reconstructor $r$. SEER is trained by jointly optimizing ${\bm \theta}_f$, ${\bm \theta}_d$, and ${\bm \theta}_r$ to minimize a weighted sum of $\mathcal{L}_{\text{nul}}$ and $\mathcal{L}_{\text{rec}}$.
• Figure 3: Example reconstructions of SEER with 128 total examples and different number of clients $C$ on CIFAR10 (Left) and 64 examples on ImageNet (Right), both using the Bright property.
• Figure 4: CIFAR10 reconstruction with secure aggregation, varying the number of clients ($C$) and total images (#Imgs). See \ref{['sec:app_full_aggr_exp']} for results with another property.
• Figure 5: SEER is robust to distribution shifts between the auxiliary dataset (CIFAR10) and the client dataset $D_c$. We use $B=128$ and the Red property.