Controllable Machine Unlearning via Gradient Pivoting

TL;DR

This work reframes machine unlearning (MU) from a single-objective optimization (SOO) problem into a multi-objective optimization (MOO) framework to explicitly balance forgetting efficacy and model fidelity. It introduces CUP, a Pivoting Gradient-based algorithm that navigates the entire Pareto frontier using a single intuitive hyperparameter, the unlearning intensity $\gamma \in [0,1]$, by rotating between two anchors in a conflict-free subspace. The authors prove convergence to Pareto-stationary points and validate CUP on both image classification and generation tasks, demonstrating superior Pareto fronts and diversity via the hypervolume indicator $\mathcal{H}$ and competitive runtime. Across CIFAR-10, SVHN, and DDPM-based generation, CUP consistently outperforms baselines in optimality (closeness to retrained performance) and controllability, enabling precise, user-guided data forgetting with practical efficiency. This approach offers a principled, data-governance-friendly pathway for trustworthy AI where fine-grained, controllable unlearning is essential.

Abstract

Machine unlearning (MU) aims to remove the influence of specific data from a trained model. However, approximate unlearning methods, often formulated as a single-objective optimization (SOO) problem, face a critical trade-off between unlearning efficacy and model fidelity. This leads to three primary challenges: the risk of over-forgetting, a lack of fine-grained control over the unlearning process, and the absence of metrics to holistically evaluate the trade-off. To address these issues, we reframe MU as a multi-objective optimization (MOO) problem. We then introduce a novel algorithm, Controllable Unlearning by Pivoting Gradient (CUP), which features a unique pivoting mechanism. Unlike traditional MOO methods that converge to a single solution, CUP's mechanism is designed to controllably navigate the entire Pareto frontier. This navigation is governed by a single intuitive hyperparameter, the `unlearning intensity', which allows for precise selection of a desired trade-off. To evaluate this capability, we adopt the hypervolume indicator, a metric that captures both the quality and diversity of the entire set of solutions an algorithm can generate. Our experimental results demonstrate that CUP produces a superior set of Pareto-optimal solutions, consistently outperforming existing methods across various vision tasks.

Figures (8)

• Figure 1: Decision boundaries of a 2-dimensional classification toy example after unlearning Class 2 with different methods: (a) Pre-trained model, (b) Gradient Ascent, (c) Weighted Scalarization, and (d) CUP.
• Figure 2: Visualization of hypervolume indicator at the two dimensional performance space.
• Figure 3: The update vector $g_t^{\text{CUP}}$ of CUP.
• Figure 4: Visualization of MNIST digit generations from a VAE model with a two-dimensional latent space after unlearning the digit ‘3’. Each point in the $(Z_1, Z_2)$ space corresponds to a generated MNIST digit.
• Figure 5: Variation of RA, UA, TA, and MIA with $\gamma$ on CIFAR-10.

Theorems & Definitions (3)

• Definition 1: Hypervolume Indicator
• Theorem 1: Convergence to Pareto-Stationary Points
• proof