Vertical Federated Unlearning via Backdoor Certification

TL;DR

The paper tackles unlearning in vertical federated learning by removing a target client’s feature contributions while preserving the rest of the collaborative model. It introduces a constrained-model gradient ascent approach, anchored by a baseline $\mathbf{W}_{\text{con}}=\frac{1}{2N-1}(2N\mathbf{W}^{(k)}-\mathbf{W}_{i}^{(k-1)})$, and bounds updates within an $\ell_2$-norm ball of radius $R$ to maintain utility. Verification is achieved via a backdoor trigger and a membership inference attack, demonstrating that the unlearned model substantially reduces the target’s influence and resists privacy attacks, with performance approaching retraining but at a fraction of the cost. Experiments on MNIST, Fashion-MNIST, and CIFAR-10 with vertical data partitioning show the method achieves comparable final accuracy to retraining, faster unlearning, and effective backdoor removal, highlighting practical viability for right-to-be-forgotten in VFL contexts.

Abstract

Vertical Federated Learning (VFL) offers a novel paradigm in machine learning, enabling distinct entities to train models cooperatively while maintaining data privacy. This method is particularly pertinent when entities possess datasets with identical sample identifiers but diverse attributes. Recent privacy regulations emphasize an individual's \emph{right to be forgotten}, which necessitates the ability for models to unlearn specific training data. The primary challenge is to develop a mechanism to eliminate the influence of a specific client from a model without erasing all relevant data from other clients. Our research investigates the removal of a single client's contribution within the VFL framework. We introduce an innovative modification to traditional VFL by employing a mechanism that inverts the typical learning trajectory with the objective of extracting specific data contributions. This approach seeks to optimize model performance using gradient ascent, guided by a pre-defined constrained model. We also introduce a backdoor mechanism to verify the effectiveness of the unlearning procedure. Our method avoids fully accessing the initial training data and avoids storing parameter updates. Empirical evidence shows that the results align closely with those achieved by retraining from scratch. Utilizing gradient ascent, our unlearning approach addresses key challenges in VFL, laying the groundwork for future advancements in this domain. All the code and implementations related to this paper are publicly available at https://github.com/mengde-han/VFL-unlearn.

Figures (6)

• Figure 1: The architecture of a vertical federated learning system
• Figure 2: The VFL framework adopted in the paper.
• Figure 3: An example of the vertically partitioned image and the implemented backdoor in MNIST
• Figure 4: Accuracy results on the clean dataset for different datasets and values of $N$.
• Figure 5: Accuracy results on the backdoor dataset for different datasets and values of $N$.