Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

An Adaptive Differentially Private Federated Learning Framework with Bi-level Optimization

Jin Wang, Hui Ma, Fei Xing, Ming Yan

TL;DR

This work tackles the joint challenges of privacy and heterogeneity in federated learning by introducing FedCompDP, an adaptive differentially private FL framework. It combines a lightweight local compression module, adaptive DP gradient clipping, and a constraint-aware robust aggregation strategy to mitigate DP-induced perturbations and Non-IID drift. Empirical results on CIFAR-10 and SVHN show improved convergence stability and higher accuracy over representative DP-FL baselines, validating the effectiveness of the three-component design. The approach offers practical privacy-preserving benefits for edge deployments and opens avenues for extending to decentralized and asynchronous FL setups.

Abstract

Federated learning enables collaborative model training across distributed clients while preserving data privacy. However, in practical deployments, device heterogeneity, non-independent, and identically distributed (Non-IID) data often lead to highly unstable and biased gradient updates. When differential privacy is enforced, conventional fixed gradient clipping and Gaussian noise injection may further amplify gradient perturbations, resulting in training oscillation and performance degradation and degraded model performance. To address these challenges, we propose an adaptive differentially private federated learning framework that explicitly targets model efficiency under heterogeneous and privacy-constrained settings. On the client side, a lightweight local compressed module is introduced to regularize intermediate representations and constrain gradient variability, thereby mitigating noise amplification during local optimization. On the server side, an adaptive gradient clipping strategy dynamically adjusts clipping thresholds based on historical update statistics to avoid over-clipping and noise domination. Furthermore, a constraint-aware aggregation mechanism is designed to suppress unreliable or noise-dominated client updates and stabilize global optimization. Extensive experiments on CIFAR-10 and SVHN demonstrate improved convergence stability and classification accuracy.

An Adaptive Differentially Private Federated Learning Framework with Bi-level Optimization

TL;DR

This work tackles the joint challenges of privacy and heterogeneity in federated learning by introducing FedCompDP, an adaptive differentially private FL framework. It combines a lightweight local compression module, adaptive DP gradient clipping, and a constraint-aware robust aggregation strategy to mitigate DP-induced perturbations and Non-IID drift. Empirical results on CIFAR-10 and SVHN show improved convergence stability and higher accuracy over representative DP-FL baselines, validating the effectiveness of the three-component design. The approach offers practical privacy-preserving benefits for edge deployments and opens avenues for extending to decentralized and asynchronous FL setups.

Abstract

Federated learning enables collaborative model training across distributed clients while preserving data privacy. However, in practical deployments, device heterogeneity, non-independent, and identically distributed (Non-IID) data often lead to highly unstable and biased gradient updates. When differential privacy is enforced, conventional fixed gradient clipping and Gaussian noise injection may further amplify gradient perturbations, resulting in training oscillation and performance degradation and degraded model performance. To address these challenges, we propose an adaptive differentially private federated learning framework that explicitly targets model efficiency under heterogeneous and privacy-constrained settings. On the client side, a lightweight local compressed module is introduced to regularize intermediate representations and constrain gradient variability, thereby mitigating noise amplification during local optimization. On the server side, an adaptive gradient clipping strategy dynamically adjusts clipping thresholds based on historical update statistics to avoid over-clipping and noise domination. Furthermore, a constraint-aware aggregation mechanism is designed to suppress unreliable or noise-dominated client updates and stabilize global optimization. Extensive experiments on CIFAR-10 and SVHN demonstrate improved convergence stability and classification accuracy.
Paper Structure (24 sections, 19 equations, 1 figure, 2 tables, 1 algorithm)

This paper contains 24 sections, 19 equations, 1 figure, 2 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. RELATED WORK
  3. Differential Privacy in Federated Learning
  4. Robust Aggregation under Non-IID Data
  5. Method
  6. Lightweight Local Compressed Module
  7. Channel Reduction and Feature Stabilization
  8. Gradient Sparsification for Structured Updates
  9. Lightweight Constraints and Device Adaptability
  10. Adaptive Differentially Private Gradient Clipping
  11. Local Gradient Clipping
  12. Adaptive Threshold Adjustment Based on Update Norms
  13. Differential Privacy Noise Injection
  14. Robust Aggregation Algorithm
  15. Local Performance and Update Norm Evaluation
  16. ...and 9 more sections

Figures (1)

  • Figure 1: Overview of the proposed differentially private federated learning framework.