Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

ERIS: Enhancing Privacy and Communication Efficiency in Serverless Federated Learning

Dario Fenoglio, Pasquale Polverino, Jacopo Quizi, Martin Gjoreski, Marc Langheinrich

TL;DR

ERIS presents a serverless federated learning framework that preserves FedAvg-level accuracy while boosting privacy and reducing communication for billion-parameter models. It achieves this through exact gradient partitioning across multiple client-side aggregators and distributed shifted gradient compression, ensuring FedAvg equivalence and information-theoretic privacy amplification. Theoretical results show convergence at the FedAvg rate and privacy leakage that decreases with the number of aggregators, while experiments across image, text, and large language models validate strong utility, substantial communication savings, and robustness to membership inference and reconstruction attacks without heavy cryptography. These properties enable scalable, privacy-preserving FL for large-scale models.

Abstract

Scaling federated learning (FL) to billion-parameter models introduces critical trade-offs between communication efficiency, model accuracy, and privacy guarantees. Existing solutions often tackle these challenges in isolation, sacrificing accuracy or relying on costly cryptographic tools. We propose ERIS, a serverless FL framework that balances privacy and accuracy while eliminating the server bottleneck and distributing the communication load. ERIS combines a model partitioning strategy, distributing aggregation across multiple client-side aggregators, with a distributed shifted gradient compression mechanism. We theoretically prove that ERIS (i) converges at the same rate as FedAvg under standard assumptions, and (ii) bounds mutual information leakage inversely with the number of aggregators, enabling strong privacy guarantees with no accuracy degradation. Experiments across image and text tasks, including large language models, confirm that ERIS achieves FedAvg-level accuracy while substantially reducing communication cost and improving robustness to membership inference and reconstruction attacks, without relying on heavy cryptography or noise injection.

ERIS: Enhancing Privacy and Communication Efficiency in Serverless Federated Learning

TL;DR

ERIS presents a serverless federated learning framework that preserves FedAvg-level accuracy while boosting privacy and reducing communication for billion-parameter models. It achieves this through exact gradient partitioning across multiple client-side aggregators and distributed shifted gradient compression, ensuring FedAvg equivalence and information-theoretic privacy amplification. Theoretical results show convergence at the FedAvg rate and privacy leakage that decreases with the number of aggregators, while experiments across image, text, and large language models validate strong utility, substantial communication savings, and robustness to membership inference and reconstruction attacks without heavy cryptography. These properties enable scalable, privacy-preserving FL for large-scale models.

Abstract

Scaling federated learning (FL) to billion-parameter models introduces critical trade-offs between communication efficiency, model accuracy, and privacy guarantees. Existing solutions often tackle these challenges in isolation, sacrificing accuracy or relying on costly cryptographic tools. We propose ERIS, a serverless FL framework that balances privacy and accuracy while eliminating the server bottleneck and distributing the communication load. ERIS combines a model partitioning strategy, distributing aggregation across multiple client-side aggregators, with a distributed shifted gradient compression mechanism. We theoretically prove that ERIS (i) converges at the same rate as FedAvg under standard assumptions, and (ii) bounds mutual information leakage inversely with the number of aggregators, enabling strong privacy guarantees with no accuracy degradation. Experiments across image and text tasks, including large language models, confirm that ERIS achieves FedAvg-level accuracy while substantially reducing communication cost and improving robustness to membership inference and reconstruction attacks, without relying on heavy cryptography or noise injection.
Paper Structure (47 sections, 8 theorems, 77 equations, 13 figures, 16 tables)

This paper contains 47 sections, 8 theorems, 77 equations, 13 figures, 16 tables.

Table of Contents

  1. Introduction
  2. Background
  3. eris
  4. Problem Definition
  5. The eris Pipeline
  6. Theoretical Analysis of Convergence and Utility
  7. Theoretical Analysis of Privacy Guarantees
  8. Results
  9. Experimental Setup
  10. Numerical Experiments
  11. Discussion
  12. Related Works
  13. Limitations and Future Works
  14. Conclusion
  15. Assumptions
  16. ...and 32 more sections

Key Result

Theorem 3.6

Consider eris under Assumptions assumption:smoothness_main and assumption:unbiased_estimator_main, where the compression operators $\mathcal{C}_k^t$ satisfy Definition def:compression_operator. Let the learning rate be defined as: where $\alpha\!=\!\frac{3\beta C_1}{2(1+\omega)L^2\theta}$, for any $\beta\!>\!0$, and let the shift stepsize be $\gamma_t\!=\!\sqrt{\frac{1+2 \omega}{2(1+\omega

Figures (13)

  • Figure 1: Illustration of eris at training round $t$ for two aggregators ($A=2$). Left: each client performs shifted compression and model partitioning, generating shards $\mathbf{v}^t_{k,(a)}$ sent to aggregators $C_2$ and $C_{k-1}$. Right: each aggregator collects and aggregates the corresponding shards across clients to produce partial updated models $\mathbf{x}^{t+1}_{(a)}$, which are then sent back to the clients.
  • Figure 2: Comparison of test accuracy and MIA accuracy across varying model capacities (one per dataset) and client-side overfitting levels, controlled via the number of training samples per client.
  • Figure 3: Effect of model partitioning (left) and shifted compression (right) on privacy.
  • Figure 4: Utility–privacy trade-off on CIFAR-10 under varying strengths of the privacy-preserving mechanisms.
  • Figure 5: Impact of honest-but-curious client collusion in eris.
  • ...and 8 more figures

Theorems & Definitions (25)

  • Remark 3.3
  • Definition 3.4: Compression operator
  • Remark 3.5
  • Theorem 3.6: Utility and communication for eris
  • Theorem 3.7: Privacy guarantee of eris
  • Remark 3.8
  • Remark 1.3
  • Theorem 2.1: Convergence equivalence of eris--Base
  • proof : Sketch
  • Remark 2.2
  • ...and 15 more