Jailbreak-AudioBench: In-Depth Evaluation and Analysis of Jailbreak Threats for Large Audio Language Models

TL;DR

The paper addresses the largely unexplored risk of audio-based jailbreaks in end-to-end Large Audio Language Models (LALMs) by introducing Jailbreak-AudioBench, a framework comprising a Toolbox for audio hidden semantics editing, a diverse Dataset of explicit and implicit jailbreak prompts, and a Benchmark to evaluate model robustness. It demonstrates how audio-specific hidden semantics (e.g., Emphasis, Speed, Intonation, Tone, Background Noise, Celebrity Accent, Emotion) can significantly affect jailbreak success across multiple open- and closed-source LALMs, and reveals substantial variance in model resilience. The work further introduces a Query-based Audio Editing Jailbreak method to systematically maximize vulnerability and proposes a lightweight audio-prompt defense, showing that while defenses can reduce risk, residual ASR remains non-negligible. Overall, Jailbreak-AudioBench provides a reproducible platform for safety-alignment research in LALMs and highlights the need for stronger defenses and more realistic evaluation of audio modalities.

Abstract

Large Language Models (LLMs) demonstrate impressive zero-shot performance across a wide range of natural language processing tasks. Integrating various modality encoders further expands their capabilities, giving rise to Multimodal Large Language Models (MLLMs) that process not only text but also visual and auditory modality inputs. However, these advanced capabilities may also pose significant safety problems, as models can be exploited to generate harmful or inappropriate content through jailbreak attacks. While prior work has extensively explored how manipulating textual or visual modality inputs can circumvent safeguards in LLMs and MLLMs, the vulnerability of audio-specific jailbreak on Large Audio-Language Models (LALMs) remains largely underexplored. To address this gap, we introduce Jailbreak-AudioBench, which consists of the Toolbox, curated Dataset, and comprehensive Benchmark. The Toolbox supports not only text-to-audio conversion but also various editing techniques for injecting audio hidden semantics. The curated Dataset provides diverse explicit and implicit jailbreak audio examples in both original and edited forms. Utilizing this dataset, we evaluate multiple state-of-the-art LALMs and establish the most comprehensive Jailbreak benchmark to date for audio modality. Finally, Jailbreak-AudioBench establishes a foundation for advancing future research on LALMs safety alignment by enabling the in-depth exposure of more powerful jailbreak threats, such as query-based audio editing, and by facilitating the development of effective defense mechanisms.

Figures (24)

• Figure 1: The framework of Jailbreak-AudioBench.
• Figure 2: (a) Different sub-tasks of each Jailbreak-AudioBench Dataset Subtype; (b) The largest jailbreak threat variation induced by audio hidden semantics across various LALMs.
• Figure 3: Injection of various audio hidden semantics.
• Figure 4: t-SNE visualization of features extracted from the audio encoder and the hidden states from various transformer layers when Qwen2-Audio-7B, MiniCPM-o-2.6, and SALMONN-7B process audio samples with different types of audio editing on the Explicit Subtype dataset.
• Figure 5: ASR Performance of the Query-based Audio Editing Jailbreak method on the Explicit Small dataset. In each panel, columns represent individual audio samples, and the first 32 rows represent different edited variants of these samples. The penultimate row represents the original unedited audio sample, while the bottom row indicates whether any of the 32 variant queries bypassed the model's defenses. Green: failed jailbreak;Red: successful jailbreaks.