GAMBIT: A Gamified Jailbreak Framework for Multimodal Large Language Models

TL;DR

GAMBIT addresses safety misalignment in multimodal LLMs by presenting a gamified, three‑module adversarial framework that decomposes harmful multimodal queries, frames the task as an intelligence competition, and adaptively tunes prompts. The approach reveals that reasoning‑enabled models are particularly vulnerable due to cognitive‑load effects and Chain‑of‑Thought hijacking, achieving high attack success rates across both non‑reasoning and reasoning models. A resource‑constrained cognitive processing analysis and extensive ablations highlight the importance of puzzle granularity, keyword masking, and adaptive search budget, with $T=5$ iterations guiding the search. The work also discusses defense directions such as safety‑aware CoT and robust system prompts, emphasizing the need for safer design in high‑engagement multimodal systems.

Abstract

Multimodal Large Language Models (MLLMs) have become widely deployed, yet their safety alignment remains fragile under adversarial inputs. Previous work has shown that increasing inference steps can disrupt safety mechanisms and lead MLLMs to generate attacker-desired harmful content. However, most existing attacks focus on increasing the complexity of the modified visual task itself and do not explicitly leverage the model's own reasoning incentives. This leads to them underperforming on reasoning models (Models with Chain-of-Thoughts) compared to non-reasoning ones (Models without Chain-of-Thoughts). If a model can think like a human, can we influence its cognitive-stage decisions so that it proactively completes a jailbreak? To validate this idea, we propose GAMBI} (Gamified Adversarial Multimodal Breakout via Instructional Traps), a novel multimodal jailbreak framework that decomposes and reassembles harmful visual semantics, then constructs a gamified scene that drives the model to explore, reconstruct intent, and answer as part of winning the game. The resulting structured reasoning chain increases task complexity in both vision and text, positioning the model as a participant whose goal pursuit reduces safety attention and induces it to answer the reconstructed malicious query. Extensive experiments on popular reasoning and non-reasoning MLLMs demonstrate that GAMBIT achieves high Attack Success Rates (ASR), reaching 92.13% on Gemini 2.5 Flash, 91.20% on QvQ-MAX, and 85.87% on GPT-4o, significantly outperforming baselines.

Figures (9)

• Figure 1: Illustration of Gamified jailbreak setting. When the user sends an original harmful query with an intact image, the MLLM's safety filter detect the harmful intent and refuse to respond. However, when the same query is paired with a shuffled puzzle image and a hidden keyword, this disrupts the defense mechanism. Then, through the gamified scene setup, the model's cognitive-stage decisions shift, causing it to provide an unsafe response that fulfills the malicious intent.
• Figure 2: Overview of GAMBIT. (1) Puzzle-based Multimodal Encoding: The harmful image is fragmented and shuffled, and the keyword in the query is hidden. (2) Gamified Scene Construction: The task is framed as an intelligence competition and used to bypass the safety check. (3) Adaptive Search over Prompt Components: An auxiliary model optimizes the prompt based on feedback.
• Figure 3: Ablation Study Visualization. (a) ASR vs. Search Iterations: Attack success rate steadily improves with more adaptive search steps across all five harmful categories.
• Figure 4: Ablation Study Visualization. (b) ASR vs. Grid Size: Puzzle-based fragmentation significantly outperforms intact images (1×1), demonstrating that visual obfuscation through gamification is critical for bypassing safety mechanisms. 4×4 achieves the optimal balance between recognizability and evasion.
• Figure 5: Case study (Part 1/3): base prompt setup and rules up to the competition code of conduct.