Understanding Data Reconstruction Leakage in Federated Learning from a Theoretical Perspective

TL;DR

This work introduces a theoretical framework to bound data reconstruction leakage in federated learning by linking attack effectiveness to the Lipschitz constant of reconstruction functions. It derives reconstruction-error bounds for FedAvg under both full and partial device participation, enabling intrinsic comparison across optimization-based DRAs. The framework is instantiated and validated on MNIST, Fashion-MNIST, and CIFAR-10 using convex FL losses, showing that attacks like InvGrad and iDLG can have inherently tighter bounds than DLG, and that GGL achieves the smallest bounds by leveraging a learned data manifold. Practically, the results provide a tool for assessing attack strength and guiding privacy defenses, with future work aimed at non-convex losses and provable defenses.

Abstract

Federated learning (FL) is an emerging collaborative learning paradigm that aims to protect data privacy. Unfortunately, recent works show FL algorithms are vulnerable to the serious data reconstruction attacks. However, existing works lack a theoretical foundation on to what extent the devices' data can be reconstructed and the effectiveness of these attacks cannot be compared fairly due to their unstable performance. To address this deficiency, we propose a theoretical framework to understand data reconstruction attacks to FL. Our framework involves bounding the data reconstruction error and an attack's error bound reflects its inherent attack effectiveness. Under the framework, we can theoretically compare the effectiveness of existing attacks. For instance, our results on multiple datasets validate that the iDLG attack inherently outperforms the DLG attack.

Figures (17)

• Figure 1: Impact of the initial parameters of a Gaussian distribution on the DRA performance. a (b) in the x-axis indicates the mean (standard deviation) of the Gaussian. The default mean and standard deviation are both 1.
• Figure 2: Iterative solvers for DRAs as unrolled deep feed-forward networks. We map the $i$-th SGD iteration in DRAs into a single network layer, and stack $I$ layers to form an $I$-layer deep network. Feeding forward data through the $I$-layer network is equivalent to executing $I$ SGD updates. The trainable parameters $\{\theta^i\}$ are colored in blue in Algorithm \ref{['alg:DRAs_gen']}, and $\theta^i$ connects the $i$-th layer and $i+1$-th layer. These parameters can be learned from intermediate reconstructed data ${\bf x}_i'$ by training the deep feed-forward network.
• Figure 3: Results of federated $\ell_2$-LogReg on MNIST---single image recovery. Dashed lines are average empirical reconstruction errors obtained by existing DRAs, while solid lines are upper bound errors obtained by our theoretical results. Y-axis is in a log form. We observe iDLG slightly outperforms DLG both empirically and theoretically; a larger $E$ and $N$ incur larger upper bound error, while a larger $T$ generates a smaller upper bound error. We have same observations on FMNIST and CIFAR10.
• Figure 4: Results of federated $\ell_2$-LogReg on FMNIST---single image recovery.
• Figure 5: Results of federated $\ell_2$-LogReg on CIFAR10---single image recovery.

Theorems & Definitions (12)

• Proposition 1
• Theorem 1
• Theorem 2
• Definition 1: Deep Feed-Forward Network
• Lemma 1
• Lemma 2: Results of one step SGD
• Lemma 3: Bounding the variance
• Lemma 4: Bounding the divergence of $\{{\bf w}_t^k\}$
• proof
• Lemma 5: Unbiased sampling scheme