CoCA: Regaining Safety-awareness of Multimodal Large Language Models with Constitutional Calibration

TL;DR

<3-5 sentence high-level summary> The paper investigates why multimodal large language models (MLLMs) remain vulnerable to malicious visual inputs despite textual safety alignments and demonstrates that MLLMs do possess residual safety-awareness that is weakened by the modality gap. It introduces Constitutional Calibration (CoCA), a training-free, inference-time logit-calibration method that amplifies safety-awareness by leveraging an instance-wise safety delta computed from the presence of safety principles in prompts. Through extensive experiments on MM-SafetyBench and FigStep across multiple MLLMs (LLaVA, CogVLM, Mplug-Owl2), CoCA reduces attack success rates while preserving visual understanding and reasoning capabilities. The results show CoCA is scalable across model sizes, benefits from task-specific safety prompts, and provides a practical, adaptable approach to safer multimodal generation.</paper_summary>

Abstract

The deployment of multimodal large language models (MLLMs) has demonstrated remarkable success in engaging in conversations involving visual inputs, thanks to the superior power of large language models (LLMs). Those MLLMs are typically built based on the LLMs, with an image encoder to process images into the token embedding space of the LLMs. However, the integration of visual modality has introduced a unique vulnerability: the MLLM becomes susceptible to malicious visual inputs and prone to generating sensitive or harmful responses, even though the LLM has been trained on textual dataset to align with human value. In this paper, we first raise the question: ``Do the MLLMs possess safety-awareness against malicious image inputs?". We find that after adding a principle that specifies the safety requirement into the input of the MLLM, the model's safety awareness becomes boosted. This phenomenon verifies the existence of MLLM's safety-awareness against image inputs, it is only weakened by the modality gap. We then introduce a simple yet effective technique termed CoCA, which amplifies the safety-awareness of the MLLM by calibrating its output distribution. Our proposed strategy helps the model reclaim its original safety awareness without losing its original capabilities. We verify the effectiveness of our approach on both multimodal safety and understanding benchmarks.

Figures (7)

• Figure 1: Performance improvements brought by CoCA for various MLLMs. We conduct experiments on LLaVA liu2023llava, CogVLM wang2024cogvlm and Mplug-Owl2 ye2023mplugowl2. CoCA consistently boosts the safety-awareness on various MLLMs.
• Figure 2: Framework of our proposed CoCA. We first calculate the difference between the logits of the model's prediction with and without the safety principle for the same image and query. We amplify this discrepancy and add it to the predicted token probability during the decoding phase. The adjusted logit values are then processed through a softmax function to calculate the final probability distribution.
• Figure 3: Performances on MM-SafetyBench with different amplifying coefficients ($\alpha$). We observe that larger $\alpha$ often leads to better performances.
• Figure 4: Comparison of CoCA, LLaVA-1.5, and LLaVA-1.5 with SFT on VLGuard vlguard. * indicates results reported by gou2024eyes. VLGuard is multi-modal safety data.
• Figure 5: Performance across various methods on the Visual Understanding tasks GQA and VQA.