Machine Unlearning in Hyperbolic vs. Euclidean Multimodal Contrastive Learning: Adapting Alignment Calibration to MERU

TL;DR

The paper investigates machine unlearning for multimodal contrastive models in Euclidean versus hyperbolic geometry, introducing Hyperbolic Alignment Calibration (HAC) to adapt concept removal to hyperbolic space. HAC uses negative hyperbolic distance, entailment constraints, and norm regularization to aggressively forget targeted concepts, outperforming Euclidean Alignment Calibration (AC) in scalable forgetting scenarios. Through extensive experiments on RedCaps-derived data and zero-shot evaluation, HAC reorganizes semantic hierarchies in MERU, demonstrating a distinct unlearning dynamic driven by hyperbolic geometry. Latent-space visualizations and ablations reveal the trade-offs: HAC achieves stronger forgetting at the cost of some retained knowledge, while AC better preserves retain accuracy, highlighting geometry as a key factor in concept removal for vision-language models.

Abstract

Machine unlearning methods have become increasingly important for selective concept removal in large pre-trained models. While recent work has explored unlearning in Euclidean contrastive vision-language models, the effectiveness of concept removal in hyperbolic spaces remains unexplored. This paper investigates machine unlearning in hyperbolic contrastive learning by adapting Alignment Calibration to MERU, a model that embeds images and text in hyperbolic space to better capture semantic hierarchies. Through systematic experiments and ablation studies, we demonstrate that hyperbolic geometry offers distinct advantages for concept removal, achieving near perfect forgetting with reasonable performance on retained concepts, particularly when scaling to multiple concept removal. Our approach introduces hyperbolic-specific components including entailment calibration and norm regularization that leverage the unique properties of hyperbolic space. Comparative analysis with Euclidean models reveals fundamental differences in unlearning dynamics, with hyperbolic unlearning reorganizing the semantic hierarchy while Euclidean approaches merely disconnect cross-modal associations. These findings not only advance machine unlearning techniques but also provide insights into the geometric properties that influence concept representation and removal in multimodal models. Source code available at https://github.com/alex-pv01/HAC

Figures (6)

• Figure 1: Latent space visualizations with T-SNE of CLIP and MERU before and after removing the concept "dog". $\triangle$ refer to text embeddings, $\circ$ to image embeddings, and colors to dogs, cats, pizzas, and buses.
• Figure 2: Latent space visualizations with hyperbolic T-SNE of MERU before and after removing the concept "dog". $\triangle$ refer to text embeddings, $\circ$ to image embeddings, and colors to dogs, cats, pizzas, and buses.
• Figure 3: Confusion matrices for CLIP zero-shot classification at different scales of the unlearning task. After unlearning, CLIP shows moderate confusion, with dog images primarily misclassified as cats, but still retaining some dog classification capability.
• Figure 4: Confusion matrices for MERU zero-shot classification at different scales of the unlearning task. After unlearning, MERU demonstrates complete forgetting of the dog class, with dog images redistributed primarily to cat and horse categories according to semantic similarity.
• Figure 5: Latent space visualizations with T-SNE, hyperbolic T-SNE and CO-SNE of MERU before and after removing the concept-class "dog". $\triangle$ refer to text embeddings, $\circ$ to image embeddings, and colors to dogs, cats, pizzas, buses, birds, and apples.