SMS: Self-supervised Model Seeding for Verification of Machine Unlearning

TL;DR

SMS (Self-supervised Model Seeding) tackles the verification gap in machine unlearning by embedding user seeds into genuine data and learning a seed-aware latent representation through a self-supervised seeding task. A seed-embedded model $M_{ exttt{S}}$ is trained jointly on the primary task and a self-supervised objective, with seed verification performed by a user-specific verifier $oldsymbol{ ext{V}}$ based on seed presence; verification bounds are given by $ ext{Pr}ig\uparrow oldsymbol{ ext{V}}(M_{ exttt{S}}, x_{i,s_i})=1ig parrow \\ge 1 - ext{eps}^{-N}$ and unambiguity bounds, while functionality is preserved so that $d(M(x), M_{ exttt{S}}(x)) \\le \

Abstract

Many machine unlearning methods have been proposed recently to uphold users' right to be forgotten. However, offering users verification of their data removal post-unlearning is an important yet under-explored problem. Current verifications typically rely on backdooring, i.e., adding backdoored samples to influence model performance. Nevertheless, the backdoor methods can merely establish a connection between backdoored samples and models but fail to connect the backdoor with genuine samples. Thus, the backdoor removal can only confirm the unlearning of backdoored samples, not users' genuine samples, as genuine samples are independent of backdoored ones. In this paper, we propose a Self-supervised Model Seeding (SMS) scheme to provide unlearning verification for genuine samples. Unlike backdooring, SMS links user-specific seeds (such as users' unique indices), original samples, and models, thereby facilitating the verification of unlearning genuine samples. However, implementing SMS for unlearning verification presents two significant challenges. First, embedding the seeds into the service model while keeping them secret from the server requires a sophisticated approach. We address this by employing a self-supervised model seeding task, which learns the entire sample, including the seeds, into the model's latent space. Second, maintaining the utility of the original service model while ensuring the seeding effect requires a delicate balance. We design a joint-training structure that optimizes both the self-supervised model seeding task and the primary service task simultaneously on the model, thereby maintaining model utility while achieving effective model seeding. The effectiveness of the proposed SMS scheme is evaluated through extensive experiments, which demonstrate that SMS provides effective verification for genuine sample unlearning, addressing existing limitations.

Figures (6)

• Figure 1: In backdooring-based methods, the backdoored data $D_b$ is mixed with the user's data $D_u$, serving distinct purposes: $D_u$ for the primary task and $D_b$ for backdooring. Hence, the removal of the backdoor can only verify the usage of the backdoored data, while it cannot confirm whether the users' genuine data has been unlearned. By contrast, our method integrates seeds into the user's genuine data, resulting in $D_{u,s}$. We will not change data labels to link and highlight the seeds as backdooring. Hence, the seeds serve as normal features of $D_{u,s}$. We designed a joint training structure to learn the seeds and primary tasks simultaneously, which links the seeds, original samples, and models for unlearning verification.
• Figure 2: Overview of the SMS Scheme. The SMS scheme consists of three main phases: (a) Users generate seeds and integrate them into their data. (b) The ML server jointly trains the model for both the primary task (green block) and the self-supervised model seeding task (blue block), with the overlapped area representing the shared network layers of the model. (c) Users verify if their seeds have disappeared from the model following unlearning requests.
• Figure 3: Evaluations of the impact of different ${\it SSR}$ on MNIST and CIFAR10
• Figure 4: Evaluations of unlearning verification performance of SMS and MIB in different unlearning scenarios using different unlearning methods. $S_1$ and $S_2$ are two kinds of seeds added to user-specified samples, $S_1$ for the erased samples and $S_2$ for other samples of his other data.
• Figure 5: The accuracy changes of SMS and MIB when executing VBU nguyen2020variational on MNIST. (a) The model accuracy diminishes rapidly on backdoored data $D_b$ while it decreases more slowly on erased data $D_e$ and test data. Since $D_e$ and $D_b$ are two independent datasets, the disappearance of the backdoored data $D_b$ does not indicate that the erased data $D_e$ has been unlearned. (b) Seeds $s_1$ and $s_2$ are embedded into the specified erased data $D_{e_1}$ and $D_{e_2}$, respectively. Both $D_{e_1,s_1}$ and $D_{e_2,s_2}$ are genuine data used for primary task training, and hence, they perform similarly during unlearning. The disappearance of seed $s_1$ indicates that the feature information of $D_{e_1,s_1}$ has been unlearned, as $s_1$ is embedded as a minor and invisible feature of $D_{e_1,s_1}$.

Theorems & Definitions (4)

• Definition 1: Verifiability
• Definition 2: Unambiguity
• Proposition 1: Strictness of SMS Verification
• proof