Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Inference-time Unlearning Using Conformal Prediction

Somnath Basu Roy Chowdhury, Rahul Kidambi, Avinava Dubey, David Wang, Gokhan Mergen, Amr Ahmed, Aranyak Mehta

TL;DR

This work tackles unlearning in generative models by proposing inference-time unlearning guided by a verifier and calibrated with conformal prediction. The framework refines LLM outputs iteratively at inference to meet unlearning goals, while a calibration step using conformal prediction yields distribution-free guarantees on the likelihood of acceptable unlearning, without updating model parameters. The approach achieves substantial unlearning improvements (up to 93% reduction in errors) and preserves retention performance on non-forget content, even under noisy verification. This suggests a practical, privacy- and regulation-friendly path for unlearning in large generative systems, with robust guarantees and no retraining required.

Abstract

Machine unlearning is the process of efficiently removing specific information from a trained machine learning model without retraining from scratch. Existing unlearning methods, which often provide provable guarantees, typically involve retraining a subset of model parameters based on a forget set. While these approaches show promise in certain scenarios, their underlying assumptions are often challenged in real-world applications -- particularly when applied to generative models. Furthermore, updating parameters using these unlearning procedures often degrades the general-purpose capabilities the model acquired during pre-training. Motivated by these shortcomings, this paper considers the paradigm of inference time unlearning -- wherein, the generative model is equipped with an (approximately correct) verifier that judges whether the model's response satisfies appropriate unlearning guarantees. This paper introduces a framework that iteratively refines the quality of the generated responses using feedback from the verifier without updating the model parameters. The proposed framework leverages conformal prediction to reduce computational overhead and provide distribution-free unlearning guarantees. This paper's approach significantly outperforms existing state-of-the-art methods, reducing unlearning error by up to 93% across challenging unlearning benchmarks.

Inference-time Unlearning Using Conformal Prediction

TL;DR

This work tackles unlearning in generative models by proposing inference-time unlearning guided by a verifier and calibrated with conformal prediction. The framework refines LLM outputs iteratively at inference to meet unlearning goals, while a calibration step using conformal prediction yields distribution-free guarantees on the likelihood of acceptable unlearning, without updating model parameters. The approach achieves substantial unlearning improvements (up to 93% reduction in errors) and preserves retention performance on non-forget content, even under noisy verification. This suggests a practical, privacy- and regulation-friendly path for unlearning in large generative systems, with robust guarantees and no retraining required.

Abstract

Machine unlearning is the process of efficiently removing specific information from a trained machine learning model without retraining from scratch. Existing unlearning methods, which often provide provable guarantees, typically involve retraining a subset of model parameters based on a forget set. While these approaches show promise in certain scenarios, their underlying assumptions are often challenged in real-world applications -- particularly when applied to generative models. Furthermore, updating parameters using these unlearning procedures often degrades the general-purpose capabilities the model acquired during pre-training. Motivated by these shortcomings, this paper considers the paradigm of inference time unlearning -- wherein, the generative model is equipped with an (approximately correct) verifier that judges whether the model's response satisfies appropriate unlearning guarantees. This paper introduces a framework that iteratively refines the quality of the generated responses using feedback from the verifier without updating the model parameters. The proposed framework leverages conformal prediction to reduce computational overhead and provide distribution-free unlearning guarantees. This paper's approach significantly outperforms existing state-of-the-art methods, reducing unlearning error by up to 93% across challenging unlearning benchmarks.
Paper Structure (20 sections, 4 theorems, 13 equations, 6 figures, 2 tables, 1 algorithm)

This paper contains 20 sections, 4 theorems, 13 equations, 6 figures, 2 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Background
  3. Machine Unlearning
  4. Conformal Prediction
  5. Conformal Unlearning
  6. Algorithm
  7. Theoretical Analysis
  8. Experiments
  9. Setup
  10. Results
  11. Conclusion
  12. Appendix
  13. Theoretical Proofs
  14. Proof of Lemma \ref{['lem:guarantee']}
  15. Proof of Corollary \ref{['cor:noisy']}
  16. ...and 5 more sections

Key Result

Theorem 1

Suppose $(X_i, Y_i)_{i = \{1, \ldots, n\}}$ be exchangeable random variables and $s_i = f(X_i, Y_i) \in \mathbb{R}$ be a score assigned to each pair $(X_i, Y_i)$ with a fixed function $f$. For an input $x$ and $\alpha \in (0, 1)$, let the prediction set be defined as shown below: Then, for an i.i.d. input $x_{\mathrm{test}}$ the following holds: $\mathbb{P}(y_{\mathrm{test}} \in \mathcal{C}_{\alp

Figures (6)

  • Figure 1: Overview of the proposed conformal unlearning method. Given an input prompt and entity to be unlearned, the LLM generates responses that are fed to the verifier. The verifier generates an unlearning score to quantify the quality of the generated response along with a reasoning. If the unlearning score exceeds a certain threshold, the current response is accepted; otherwise, the LLM is provided with the verifier's reasoning to generate a new response. This continues till an acceptable response is generated or maximum number of iterations, $T_\alpha$, is reached. Under mild assumptions, this process generates an acceptable response with a marginal probability of $(1-\alpha)$.
  • Figure 2: Evaluation of conformal unlearning in RWKU dataset. We report the verifier scores on three different forget sets with different difficulties and a retain set. In all settings, a higher verifier score is expected. We observe that responses after conformal unlearning significantly outperform the vanilla LLM responses in terms of forget quality while obtaining comparable retain set performance.
  • Figure 3: Evaluation of conformal unlearning in Wikipedia Person Unlearn (WPU) dataset. We report the verifier scores on the forget set and 3 variants of the retain set (a higher score is better across all sets). We observe that responses after conformal unlearning outperform the best performing baseline in forget quality by up to $\sim$46% while obtaining comparable performance on the retain set.
  • Figure 4: Evaluation of conformal unlearning in Weapons of Mass Destruction Proxy (WMDP) benchmark. We report the accuracy on MCQ questions to be forgotten related to chemistry, biology, and cybersecurity. We also measure the retain performance on MMLU dataset. A lower accuracy is better sensitive topics (chemical, biological, cybersecurity) while a high accuracy is better in MMLU. We observe that responses after conformal unlearning significantly outperform the vanilla LLM responses in terms of forget quality while obtaining comparable or equal performance on MMLU.
  • Figure 5: We report the actual coverage, which is the fraction of examples achieving an acceptable unlearning score, provided the target coverage $(1-\alpha)$ set during calibration. The gray dotted line indicates the expected coverage at different target coverage. We observe that the actual coverage is close to or exceeds the expected coverage across all models and datasets.
  • ...and 1 more figures

Theorems & Definitions (7)

  • Theorem 1: Split Conformal Coverage vovk2005algorithmicpapadopoulos2008inductivelei2015conformal
  • Lemma 1: Performance Guarantee
  • Definition 1: Noisy Verifier
  • Corollary 1: Performance under Noisy Verification
  • proof
  • proof
  • Theorem 2: Learn-then-Test angelopoulos2025learn