FUPareto: Bridging the Forgetting-Utility Gap in Federated Unlearning via Pareto Augmented Optimization

TL;DR

FUPareto addresses the forgetting-utility trade-off in Federated Unlearning by framing it as a multi-objective problem and solving it with Pareto-augmented optimization. It introduces the Minimum Boundary Shift loss to achieve forgetting with minimal parameter changes and MIA risk, then alternates between Pareto improvement and Pareto expansion using MGDA, with a null-space projection to decouple gradient conflicts. A recovery-oriented post-training stage with an anchor constraint further recovers retained utility without reintroducing forgetting pressure. Empirical results across multiple datasets show strong forgetting efficacy, robust utility retention, and improved privacy robustness over state-of-the-art baselines, highlighting its practicality for concurrent multi-client unlearning in privacy-conscious federated systems.

Abstract

Federated Unlearning (FU) aims to efficiently remove the influence of specific client data from a federated model while preserving utility for the remaining clients. However, three key challenges remain: (1) existing unlearning objectives often compromise model utility or increase vulnerability to Membership Inference Attacks (MIA); (2) there is a persistent conflict between forgetting and utility, where further unlearning inevitably harms retained performance; and (3) support for concurrent multi-client unlearning is poor, as gradient conflicts among clients degrade the quality of forgetting. To address these issues, we propose FUPareto, an efficient unlearning framework via Pareto-augmented optimization. We first introduce the Minimum Boundary Shift (MBS) Loss, which enforces unlearning by suppressing the target class logit below the highest non-target class logit; this can improve the unlearning efficiency and mitigate MIA risks. During the unlearning process, FUPareto performs Pareto improvement steps to preserve model utility and executes Pareto expansion to guarantee forgetting. Specifically, during Pareto expansion, the framework integrates a Null-Space Projected Multiple Gradient Descent Algorithm (MGDA) to decouple gradient conflicts. This enables effective, fair, and concurrent unlearning for multiple clients while minimizing utility degradation. Extensive experiments across diverse scenarios demonstrate that FUPareto consistently outperforms state-of-the-art FU methods in both unlearning efficacy and retained utility.

Figures (5)

• Figure 1: Gradient conflicts and the trade-off between forgetting and utility. Left: Gradient interactions between unlearning and remaining clients. At round $t$, the unlearning gradient $g_u^t$ and the remaining gradient $g_r^t$ are not conflict (i.e., $g_u^t \cdot g_r^t > 0$) or exhibit mild conflict, yielding a common descent direction $g_d^t$ that simultaneously minimizes both objectives. Conversely, at round $t+1$, severe gradient conflict negates this direction as the model approaches a local Pareto stationary point. Right: Evolution in the objective space. The updated $F(w^{t+1})$ propels the model toward a local Pareto frontier, a boundary where further decreasing $F_u$ inevitably degrades $F_r$. The orange trajectory highlights the Pareto expansion in FUPareto, which temporarily deviates from the frontier to facilitate deeper unlearning.
• Figure 2: Overview of the FUPareto framework, illustrated with one unlearning client and one remaining client. The method comprises two stages: (a) unlearning stage, which alternates between Pareto improvement and expansion to remove target data while preserving utility, and (b) post-training with anchor-constrained multi-objective improvement to improve performance of retained clients.
• Figure 3: Illustration of the Pareto improvement step in a three-client scenario, involving unlearning clients $u_1, u_2$ and a remaining client $r$. MGDA derives a common descent direction to steer the model toward the Pareto frontier, indicated by the black arrow. Incorporating the fairness-guidance objective $g_p^t$ yields a fairer solution where the discrepancy between $F_{u_1}$ and $F_{u_2}$ is minimized, as marked by the red point.
• Figure 4: ASR, R-Acc, and distance from $\omega^0$ during unlearning and post-training on CIFAR-10 with 2 of 10 clients unlearning.
• Figure 5: AUC of each unlearning client on (a) FMNIST and (b) CIFAR-10.