FedShard: Federated Unlearning with Efficiency Fairness and Performance Fairness

TL;DR

FedShard tackles federated unlearning by enforcing efficiency fairness and performance fairness through a novel sharded framework. It introduces two adaptive algorithms—shard merging for directional diversity and variance-aware training round allocation—to ensure fair and efficient unlearning while preserving convergence. Theoretical results prove exact unlearning and quantify speedups, while extensive experiments demonstrate 1.3–6.2x faster unlearning than retraining and up to 4.9x faster than state-of-the-art exact methods, along with favorable fairness metrics $M_p$ and $M_e$. FedShard also mitigates cascaded leaving and poisoning risks, delivering balanced unlearning costs and robust model performance across diverse non-IID settings and datasets.

Abstract

To protect clients' right to be forgotten in federated learning, federated unlearning aims to remove the data contribution of leaving clients from the global learned model. While current studies mainly focused on enhancing unlearning efficiency and effectiveness, the crucial aspects of efficiency fairness and performance fairness among decentralized clients during unlearning have remained largely unexplored. In this study, we introduce FedShard, the first federated unlearning algorithm designed to concurrently guarantee both efficiency fairness and performance fairness. FedShard adaptively addresses the challenges introduced by dilemmas among convergence, unlearning efficiency, and unlearning fairness. Furthermore, we propose two novel metrics to quantitatively assess the fairness of unlearning algorithms, which we prove to satisfy well-known properties in other existing fairness measurements. Our theoretical analysis and numerical evaluation validate FedShard's fairness in terms of both unlearning performance and efficiency. We demonstrate that FedShard mitigates unfairness risks such as cascaded leaving and poisoning attacks and realizes more balanced unlearning costs among clients. Experimental results indicate that FedShard accelerates the data unlearning process 1.3-6.2 times faster than retraining from scratch and 4.9 times faster than the state-of-the-art exact unlearning methods.

Figures (16)

• Figure 1: An illustration of performance unfairness and efficiency unfairness in federated unlearning. Red exclamation marks indicate that clients encounter unfairness.
• Figure 2: Left: A conceptual illustration showing that only FedShard simultaneously achieves both performance fairness and efficiency fairness. Right: An example of the FedShard architecture with a merging rate of 3. Clients are organized in a tree-like structure. When clients request unlearning (indicated in blue), only the shards on the direct path to the root (dark background) require retraining.
• Figure 3: Illustration of the FedShard learning and unlearning process of the example in Figure \ref{['fig:arch1']}. Each node represents a shard, and each layer presents a training stage. Arrows indicate the merging of shards between stages. When client $c_{25}$ requests unlearning, only the shards on its hierarchical path (dark background) require retraining.
• Figure 4: Example of the shard merging strategy. To form a new shard for the next stage, FedShard combines shards with diverse update directions (represented by arrows) to promote stable convergence.
• Figure 5: Global model accuracy on CIFAR-10 (Non-IID $\rho=0.1$). The standard FedAvg (a) gives a baseline model performance on the federated training. The ablation study shows that naive sharding (b) is unstable. Our merging algorithm $\mathcal{A}_1$ (c) stabilizes training, and our round allocation algorithm $\mathcal{A}_2$ (d) further enables FedShard to surpass the accuracy of standard FedAvg.

Theorems & Definitions (29)

• Proposition 1
• Proposition 2
• Proposition 3: Efficiency in the General Case
• Proposition 4
• Corollary 1
• proof
• Proposition 5
• proof
• Proposition 6
• proof