Unlearning Clients, Features and Samples in Vertical Federated Learning

TL;DR

The paper tackles unlearning in vertical federated learning by introducing two approaches: VFU-KD, a knowledge distillation–based method for passive party and feature unlearning, and VFU-GA, a gradient ascent–based method for sample unlearning. Both methods aim to minimize communication between active and passive parties while preserving utility, with the active party storing previous embeddings. Unlearning effectiveness is audited using a membership inference attack, enabling verification without compromising VFL constraints. Empirical results on six tabular and two image datasets show that VFU-KD and VFU-GA achieve comparable or superior performance to retraining from scratch and R2S baselines, with modest utility losses in some cases and clear advantages in terms of communication efficiency. The work outlines limitations and proposes future directions, including stronger auditing models and storage optimizations for practical deployment.

Abstract

Federated Learning (FL) has emerged as a prominent distributed learning paradigm. Within the scope of privacy preservation, information privacy regulations such as GDPR entitle users to request the removal (or unlearning) of their contribution from a service that is hosting the model. For this purpose, a server hosting an ML model must be able to unlearn certain information in cases such as copyright infringement or security issues that can make the model vulnerable or impact the performance of a service based on that model. While most unlearning approaches in FL focus on Horizontal FL (HFL), where clients share the feature space and the global model, Vertical FL (VFL) has received less attention from the research community. VFL involves clients (passive parties) sharing the sample space among them while not having access to the labels. In this paper, we explore unlearning in VFL from three perspectives: unlearning clients, unlearning features, and unlearning samples. To unlearn clients and features we introduce VFU-KD which is based on knowledge distillation (KD) while to unlearn samples, VFU-GA is introduced which is based on gradient ascent. To provide evidence of approximate unlearning, we utilize Membership Inference Attack (MIA) to audit the effectiveness of our unlearning approach. Our experiments across six tabular datasets and two image datasets demonstrate that VFU-KD and VFU-GA achieve performance comparable to or better than both retraining from scratch and the benchmark R2S method in many cases, with improvements of $(0-2\%)$. In the remaining cases, utility scores remain comparable, with a modest utility loss ranging from $1-5\%$. Unlike existing methods, VFU-KD and VFU-GA require no communication between active and passive parties during unlearning. However, they do require the active party to store the previously communicated embeddings.

Figures (16)

• Figure 1: Vertical federated learning framework
• Figure 2: MIA attack model.
• Figure 3: The training and test loss of VFU-KD compared to the retrained model from scratch and R2S method.
• Figure 4: The training (red) and test loss (blue) of VFU-KD (solid lines) with $\mathcal{H}^{-1}$ compared to the retrained model from scratch (dotted lines).
• Figure 5: The loss curves of VFU-KD compared to the retrained model from scratch and R2S method.