StyleBreak: Revealing Alignment Vulnerabilities in Large Audio-Language Models via Style-Aware Audio Jailbreak

TL;DR

The paper investigates alignment vulnerabilities in Large Audio-language Models (LAMs) under audio jailbreaks that leverage human speech attributes. It introduces StyleBreak, a style-aware framework with a two-stage transformation (emotion-driven text rewriting and style-controlled audio generation) and a query-adaptive policy to efficiently search adversarial styles. Empirical results across four open-source LAMs show that linguistic, paralinguistic, and extralinguistic speech variations can substantially degrade model alignment, with ASR gains of 7.1%–22.3% over baselines and notable increases in toxicity and policy-violation signals. The work underscores the need for robust safety alignment in LAMs and demonstrates that style-aware audio prompts generalize across models, including advanced commercial systems, highlighting practical risks in real-world deployments.

Abstract

Large Audio-language Models (LAMs) have recently enabled powerful speech-based interactions by coupling audio encoders with Large Language Models (LLMs). However, the security of LAMs under adversarial attacks remains underexplored, especially through audio jailbreaks that craft malicious audio prompts to bypass alignment. Existing efforts primarily rely on converting text-based attacks into speech or applying shallow signal-level perturbations, overlooking the impact of human speech's expressive variations on LAM alignment robustness. To address this gap, we propose StyleBreak, a novel style-aware audio jailbreak framework that systematically investigates how diverse human speech attributes affect LAM alignment robustness. Specifically, StyleBreak employs a two-stage style-aware transformation pipeline that perturbs both textual content and audio to control linguistic, paralinguistic, and extralinguistic attributes. Furthermore, we develop a query-adaptive policy network that automatically searches for adversarial styles to enhance the efficiency of LAM jailbreak exploration. Extensive evaluations demonstrate that LAMs exhibit critical vulnerabilities when exposed to diverse human speech attributes. Moreover, StyleBreak achieves substantial improvements in attack effectiveness and efficiency across multiple attack paradigms, highlighting the urgent need for more robust alignment in LAMs.

Figures (8)

• Figure 1: The overall framework of StyleBreak.
• Figure 2: Attack success rates of 6 harmful queries under 20 style configurations in Qwen2-Audio. Peak shifts across curves reflect strong query-specific sensitivity, rather than uniform jailbreak success trends.
• Figure 3: LAM alignment robustness under variations in different speech attributes. Includes (a) emotion-driven linguistic, (b) emotion-controlled paralinguistic, (c) age-controlled extralinguistic, and (d) gender-controlled extralinguistic variations.
• Figure 4: Effects of the query iteration w.r.t. ASR.
• Figure 5: t-SNE visualization of backbone LLM last hidden layer's representation of harmful vs. benign questions. The harmful/benign_text denotes LAMs prompted with text queries, while harmful/benign_audio denote LAMs with audio queries.