Missing-by-Design: Certifiable Modality Deletion for Revocable Multimodal Sentiment Analysis

TL;DR

Missing-by-Design (MBD) is presented, a unified framework for revocable multimodal sentiment analysis that combines structured representation learning with a certifiable parameter-modification pipeline, positioning surgical unlearning as an efficient alternative to full retraining.

Abstract

As multimodal systems increasingly process sensitive personal data, the ability to selectively revoke specific data modalities has become a critical requirement for privacy compliance and user autonomy. We present Missing-by-Design (MBD), a unified framework for revocable multimodal sentiment analysis that combines structured representation learning with a certifiable parameter-modification pipeline. Revocability is critical in privacy-sensitive applications where users or regulators may request removal of modality-specific information. MBD learns property-aware embeddings and employs generator-based reconstruction to recover missing channels while preserving task-relevant signals. For deletion requests, the framework applies saliency-driven candidate selection and a calibrated Gaussian update to produce a machine-verifiable Modality Deletion Certificate. Experiments on benchmark datasets show that MBD achieves strong predictive performance under incomplete inputs and delivers a practical privacy-utility trade-off, positioning surgical unlearning as an efficient alternative to full retraining.

Figures (6)

• Figure 1: Overview of the Missing-by-Design (MBD) framework for certifiable modality deletion. The architecture is organized into two primary stages: a property-informed joint training phase and a weight surgery pipeline. In the training phase, multimodal inputs ($X^L, X^A, X^V$) are integrated via a fusion network $\mathcal{F}$ for sentiment prediction, while auxiliary generator networks $\mathcal{G}_m$ and property embeddings $P^m$ are optimized to enforce modality-specific reconstruction and property alignment. Upon a revocation request for modality $m^\star$, the surgery pipeline utilizes a calibration batch $\mathcal{B}$ to compute the modality saliency $s_q^{(m^\star)}$ and a SwiftPrune-inspired importance proxy $L_q$. These metrics guide the candidate selection and thresholding process ($\eta_s, \eta_L$), followed by a differential-privacy calibrated Gaussian mechanism ($\varepsilon_{\mathrm{mod}}, \delta_{\mathrm{mod}}$) for parameter modification. The pipeline ultimately outputs the modified model parameters $\mathcal{W}'$ alongside a machine-verifiable Modality Deletion Certificate (MDC).
• Figure 2: Privacy–utility trade-off after certified audio deletion. Plotted curves show binary accuracy (Acc2) together with attack success rate (ASR, white-box) as functions of $\varepsilon_{\mathrm{mod}}$. Lower ASR and higher Acc2 are preferred.
• Figure 3: Training trajectories for the principal loss terms (averaged across three seeds).
• Figure 4: t-SNE visualization of reconstructed embeddings (left: without property embedding pathway; right: with property embedding pathway).
• Figure 5: Cumulative privacy budget under sequential modality deletions. The solid curve shows the theoretical $(\varepsilon,\delta)$ conversion obtained from zCDP composition, while the dashed curve reports the empirical quantile estimated by Monte Carlo sampling of the privacy-loss random variable.

Theorems & Definitions (3)

• Theorem A.1: DP-like indistinguishability
• Lemma A.2: Pointwise proxy error bound
• proof