Secure Generalization through Stochastic Bidirectional Parameter Updates Using Dual-Gradient Mechanism

TL;DR

The paper addresses privacy leakage in federated learning by introducing stochastic bidirectional parameter updates (SBPU) that generate diverse yet closely situated global models at the server. SBPU uses a dual-gradient mechanism to perturb gradients in a fine-grained, layer-wise manner, creating neighborhoods of solutions that improve generalization while strengthening resistance to attacks such as LIA, MIA, and IR, without sharing classifier parameters. The method is embedded in an encrypted-domain GAN framework with a generator G, discriminator D, feature extractor F, and classifier C, optimized via a composite loss that includes adversarial, semantic, and classification components. Empirical results on MNIST, FMNIST, CIFAR10, and SVHN demonstrate state-of-the-art utility and robustness against privacy attacks, outperforming PPIDSG and related baselines while maintaining practical training times and scalability.

Abstract

Federated learning (FL) has gained increasing attention due to privacy-preserving collaborative training on decentralized clients, mitigating the need to upload sensitive data to a central server directly. Nonetheless, recent research has underscored the risk of exposing private data to adversaries, even within FL frameworks. In general, existing methods sacrifice performance while ensuring resistance to privacy leakage in FL. We overcome these issues and generate diverse models at a global server through the proposed stochastic bidirectional parameter update mechanism. Using diverse models, we improved the generalization and feature representation in the FL setup, which also helped to improve the robustness of the model against privacy leakage without hurting the model's utility. We use global models from past FL rounds to follow systematic perturbation in parameter space at the server to ensure model generalization and resistance against privacy attacks. We generate diverse models (in close neighborhoods) for each client by using systematic perturbations in model parameters at a fine-grained level (i.e., altering each convolutional filter across the layers of the model) to improve the generalization and security perspective. We evaluated our proposed approach on four benchmark datasets to validate its superiority. We surpassed the state-of-the-art methods in terms of model utility and robustness towards privacy leakage. We have proven the effectiveness of our method by evaluating performance using several quantitative and qualitative results.

Figures (5)

• Figure 1: Visualization of diverse global models obtained using proposed approach ($SBPU$). We generate diverse models in close neighborhoods through systematic updates. It is important to note that we do not make perturbations after generating the diverse models, which helps to retain utility and generalization.
• Figure 2: The schematic architecture diagram of the proposed approach.
• Figure 3: High-level representation of different components in the overall architecture.
• Figure 4: Comparative analysis under reconstruction attack. Here, a small PSNR value denotes privacy preservation, i.e., the robustness of the model against IR attack. Through histogram plots, we can see that the generated image for a given CIFAR10 sample is encrypted and does not reveal visual information (either visually or through pixel distribution), which affirms the effectiveness of our method.
• Figure 5: Comparison with PPIDSG using CIFAR10 and FMNIST.