Reimagining Safety Alignment with An Image

TL;DR

This work addresses jailbreak attacks and over-refusal in multimodal LLMs by proposing Magic Image, an optimization-driven, parameter-free visual prompt that learns a perturbation $x_\text{MI}$ initialized as a white image to modulate model behavior without updating weights. It defines formal problems for jailbreak and over-refusal using $P_\theta(r_{1:k}|x_\text{jail})$ and $X_\text{OR}$ with threshold $\gamma$, and optimizes a dual loss $\mathcal{L}(\text{dual})$ with $\lambda_1+\lambda_2=1$ to balance jailbreak defense and benign acceptance. The approach is validated across three MLLMs and five datasets, demonstrating superior safety-efficiency balance (SE-score) while preserving benign performance, and showing transferability to unseen data. This lightweight, image-based control mechanism offers scalable adaptability to different safety preferences and regulatory environments, addressing a practical gap in deployable MLLM safety alignment.

Abstract

Large language models (LLMs) excel in diverse applications but face dual challenges: generating harmful content under jailbreak attacks and over-refusal of benign queries due to rigid safety mechanisms. These issues are further complicated by the need to accommodate different value systems and precisely align with given safety preferences. Moreover, traditional methods like SFT and RLHF lack this capability due to their costly parameter tuning requirements and inability to support multiple value systems within a single model. These problems are more obvious in multimodal large language models (MLLMs), especially in terms of heightened over-refusal in cross-modal tasks and new security risks arising from expanded attack surfaces. We propose Magic Image, an optimization-driven visual prompt framework that enhances security while reducing over-refusal. By optimizing image prompts using harmful/benign samples, our method enables a single model to adapt to different value systems and better align with given safety preferences without parameter updates. Experiments demonstrate improved safety-effectiveness balance across diverse datasets while preserving model performance, offering a practical solution for deployable MLLM safety alignment.

Figures (10)

• Figure 1: The overview of Magic Image. We construct jailbreak data and borderline data that contain contextual and few-shot prompts, use the target model to generate responses, and update the model by comparing target responses via cross-entropy loss. Ultimately, this method effectively enhances the model's robustness against jailbreak data while maintaining normal responsiveness to borderline data.
• Figure 2: Comparison of the refuse rate of the Llava-v1.6-mistral model with and without a plain white image added to the text input. Text-image input changes the model output distribution, demonstrating that visual information can guide the model in distinguishing input sample types.
• Figure 3: How Magic Image influences the distribution of borderline data and jailbreak data in the model's decision space. Magic Image can correct misclassified inputs while maintaining the decisions for normal samples unchanged.
• Figure 4: Specific examples of different prompt strategies
• Figure 5: Invalid responses from SCANS for some queries