ModalImmune: Immunity Driven Unlearning via Self Destructive Training

TL;DR

Empirical evaluation on standard multimodal benchmarks demonstrates that ModalImmune improves resilience to modality removal and corruption while retaining convergence stability and reconstruction capacity.

Abstract

Multimodal systems are vulnerable to partial or complete loss of input channels at deployment, which undermines reliability in real-world settings. This paper presents ModalImmune, a training framework that enforces modality immunity by intentionally and controllably collapsing selected modality information during training so the model learns joint representations that are robust to destructive modality influence. The framework combines a spectrum-adaptive collapse regularizer, an information-gain guided controller for targeted interventions, curvature-aware gradient masking to stabilize destructive updates, and a certified Neumann-truncated hyper-gradient procedure for automatic meta-parameter adaptation. Empirical evaluation on standard multimodal benchmarks demonstrates that ModalImmune improves resilience to modality removal and corruption while retaining convergence stability and reconstruction capacity.

Figures (8)

• Figure 1: Overview of the ModalImmune framework, which treats modality destruction as an active causal intervention. The training strategy alternates between standard reconstruction and Self-Destructive Learning (SDL) built on three key components: Info-Drop Intervention (IDI), where an EXP3.P bandit controller leverages an information-gain surrogate $\ell_m$ to adaptively select the target modality $m^\star$; Spectral Self-Collapse (SSC), enforcing an irreversible directional loss through a spectrum-adaptive regularizer $L_{\mathrm{coll}}$ combined with a stable-rank penalty; Curvature-Gated Counter-Gradient (CGC), which inspects empirical Fisher information to apply a masking multiplier $\rho$ that mitigates destabilizing gradient ascent. Meanwhile, meta-parameters $\xi = \{\lambda,\eta,\kappa\}$ are optimized via Bi-level Hyper-Gradient Descent (BHGD) using a certified Neumann-truncated estimator. This unified protocol ensures the Fusion Hub and Task Output$(C)$ remain robust under counterfactual interventions and adversarial modality collapse.
• Figure 2: Training dynamics with explicit phase markers. The horizontal axis shows epochs from 0 to 50. The left vertical axis reports validation Acc2 and the right vertical axis reports MAE. Background shading separates Self-Destructive Learning (SDL) phases from reconstruction-only phases. Shaded bands indicate the 95% confidence interval over three independent seeds. The plot demonstrates that introducing SDL phases does not slow convergence nor induce overfitting on the validation set.
• Figure 3: Quantified contribution of principal modules. Bars show absolute drops in validation Acc2 (percentage points) when the named module is ablated. The property-vector pathway produces the largest single decrease, indicating its central role in the model's predictive power. This visual summary complements the ablation table by converting numeric differences into an immediately interpretable ranking.
• Figure 4: BHGD hyperparameter trajectories versus grid-search baselines. Each subplot shows the online evolution of one meta-parameter across training epochs: collapse weight $\lambda$, negative feedback scale $\kappa$, and stable-rank penalty $\eta$. Horizontal dashed lines mark best values found by offline grid search. An inset table compares wall-clock time for hyperparameter selection and demonstrates that BHGD attains comparable or superior meta-parameters while reducing tuning wall time by approximately 38%.
• Figure 5: Certified Neumann truncation: error versus compute. The horizontal axis shows truncation depth $K$ from 1 to 10. The left vertical axis reports the relative hyper-gradient error norm $\lVert\widehat{\nabla}-\nabla\rVert/\lVert\nabla\rVert$ expressed in percent. The right vertical axis reports the average additional training time in seconds per hyper-update. A vertical dashed line indicates the online doubling stopping point (here $K=6$), at which point the relative error falls below 2% while compute remains practical. The plot supports the claim that the chosen truncation rule yields an efficient and accurate approximation.

Theorems & Definitions (12)

• Proposition A.1
• proof
• Lemma A.2
• proof
• Proposition A.3
• proof
• Theorem A.4
• proof
• Proposition A.5
• proof