Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

MultiDelete for Multimodal Machine Unlearning

Jiali Cheng, Hadi Amiri

TL;DR

This paper addresses the challenge of unlearning specific multimodal training data without full retraining by introducing MultiDelete, a modality-aware framework that decouples inter-modal relationships while preserving both multimodal and unimodal knowledge. It formalizes three core properties—modality decoupling, multimodal knowledge retention, and unimodal knowledge retention—and combines corresponding losses into a unified optimization objective. Across image-text and graph-text tasks, MultiDelete outperforms modality-agnostic and unimodal baselines in forgetting deleted data and maintaining task performance, while reducing membership inference risk. The approach demonstrates notable efficiency and robustness, suggesting practical impact for privacy-preserving updates in large multimodal models.

Abstract

Machine Unlearning removes specific knowledge about training data samples from an already trained model. It has significant practical benefits, such as purging private, inaccurate, or outdated information from trained models without the need for complete re-training. Unlearning within a multimodal setting presents unique challenges due to the complex dependencies between different data modalities and the expensive cost of training on large multimodal datasets and architectures. This paper presents the first machine unlearning approach for multimodal data and models, titled MultiDelete, which is designed to decouple associations between unimodal data points during unlearning without losing the overall representation strength of the trained model. MultiDelete advocates for three key properties for effective multimodal unlearning: (a): modality decoupling, which effectively decouples the association between individual unimodal data points marked for deletion, rendering them as unrelated data points, (b): multimodal knowledge retention, which retains the multimodal representation post-unlearning, and (c): unimodal knowledge retention, which retains the unimodal representation postunlearning. MultiDelete is efficient to train and is not constrained by using a strongly convex loss -- a common restriction among existing baselines. Experiments on two architectures and four datasets, including image-text and graph-text datasets, show that MultiDelete gains an average improvement of 17.6 points over best performing baseline in unlearning multimodal samples, can maintain the multimodal and unimodal knowledge of the original model post unlearning, and can provide better protection to unlearned data against adversarial attacks.

MultiDelete for Multimodal Machine Unlearning

TL;DR

This paper addresses the challenge of unlearning specific multimodal training data without full retraining by introducing MultiDelete, a modality-aware framework that decouples inter-modal relationships while preserving both multimodal and unimodal knowledge. It formalizes three core properties—modality decoupling, multimodal knowledge retention, and unimodal knowledge retention—and combines corresponding losses into a unified optimization objective. Across image-text and graph-text tasks, MultiDelete outperforms modality-agnostic and unimodal baselines in forgetting deleted data and maintaining task performance, while reducing membership inference risk. The approach demonstrates notable efficiency and robustness, suggesting practical impact for privacy-preserving updates in large multimodal models.

Abstract

Machine Unlearning removes specific knowledge about training data samples from an already trained model. It has significant practical benefits, such as purging private, inaccurate, or outdated information from trained models without the need for complete re-training. Unlearning within a multimodal setting presents unique challenges due to the complex dependencies between different data modalities and the expensive cost of training on large multimodal datasets and architectures. This paper presents the first machine unlearning approach for multimodal data and models, titled MultiDelete, which is designed to decouple associations between unimodal data points during unlearning without losing the overall representation strength of the trained model. MultiDelete advocates for three key properties for effective multimodal unlearning: (a): modality decoupling, which effectively decouples the association between individual unimodal data points marked for deletion, rendering them as unrelated data points, (b): multimodal knowledge retention, which retains the multimodal representation post-unlearning, and (c): unimodal knowledge retention, which retains the unimodal representation postunlearning. MultiDelete is efficient to train and is not constrained by using a strongly convex loss -- a common restriction among existing baselines. Experiments on two architectures and four datasets, including image-text and graph-text datasets, show that MultiDelete gains an average improvement of 17.6 points over best performing baseline in unlearning multimodal samples, can maintain the multimodal and unimodal knowledge of the original model post unlearning, and can provide better protection to unlearned data against adversarial attacks.
Paper Structure (34 sections, 7 equations, 2 figures, 8 tables)

This paper contains 34 sections, 7 equations, 2 figures, 8 tables.

Table of Contents

  1. Introduction
  2. MultiDelete
  3. Notation
  4. Problem Formulation
  5. Key Properties for Multimodal Unlearning
  6. Modality Decoupling
  7. Multimodal Knowledge Retention
  8. Unimodal Knowledge Retention
  9. Optimization
  10. Experimental Setup
  11. Tasks and Datasets
  12. Baselines
  13. Evaluation
  14. Settings
  15. Multimodal Architectures
  16. ...and 19 more sections

Figures (2)

  • Figure 1: Summary of the proposed approach, MultiDelete. (a) given a trained multimodal model (e.g. a vision-language model) and a subset of its training data $D_f$ with image-text relations marked for unlearning or deletion, MultiDelete decouples the inter-modality dependency on $D_f$, while maintaining the multimodal and unimodal knowledge on the remaining training data $D_r = D_{\mathrm{train}} \setminus D_f$. (b) modality decoupling: ensures that individual modalities in the deleted data pairs $D_f$ are treated as unrelated by the model, (c) multimodal knowledge retention: preserves the model's overall multimodal knowledge, and (d) unimodal knowledge retention: preserves the model's understanding of individual modalities.
  • Figure 2: Training time of unlearning methods.

Theorems & Definitions (3)

  • definition thmcounterdefinition: modality decoupling
  • definition thmcounterdefinition: multimodal knowledge retention
  • definition thmcounterdefinition: unimodal knowledge retention