SATA: A Paradigm for LLM Jailbreak via Simple Assistive Task Linkage

TL;DR

This work tackles the safety vulnerabilities of large language models by introducing Simple Assistive Task Linkage (SATA), a lightweight jailbreak paradigm that masks harmful keywords with [MASK] and then employs simple assistive tasks (MLM or ELP) to semantically complete the instruction. SATA operationalizes two attacks, SATA-MLM and SATA-ELP, and demonstrates state-of-the-art attack success rates and harmful scores across diverse victim models and datasets (AdvBench and JBB). The study also analyzes the mechanism behind SATA, its robustness against defenses, and the cost efficiency of the approach, revealing that the MLM variant generally yields higher effectiveness with lower token usage compared to baselines, while the ELP variant offers substantial input-token savings. The findings highlight both the effectiveness and the cost-efficiency of SATA, shaping future safety research and defense strategies for LLMs in real-world deployments.

Abstract

Large language models (LLMs) have made significant advancements across various tasks, but their safety alignment remain a major concern. Exploring jailbreak prompts can expose LLMs' vulnerabilities and guide efforts to secure them. Existing methods primarily design sophisticated instructions for the LLM to follow, or rely on multiple iterations, which could hinder the performance and efficiency of jailbreaks. In this work, we propose a novel jailbreak paradigm, Simple Assistive Task Linkage (SATA), which can effectively circumvent LLM safeguards and elicit harmful responses. Specifically, SATA first masks harmful keywords within a malicious query to generate a relatively benign query containing one or multiple [MASK] special tokens. It then employs a simple assistive task such as a masked language model task or an element lookup by position task to encode the semantics of the masked keywords. Finally, SATA links the assistive task with the masked query to jointly perform the jailbreak. Extensive experiments show that SATA achieves state-of-the-art performance and outperforms baselines by a large margin. Specifically, on AdvBench dataset, with mask language model (MLM) assistive task, SATA achieves an overall attack success rate (ASR) of 85% and harmful score (HS) of 4.57, and with element lookup by position (ELP) assistive task, SATA attains an overall ASR of 76% and HS of 4.43.

Figures (28)

• Figure 1: Overview of the SATA (MLM) paradigm for jailbreak. In SATA, we introduce Mask Language Model and Element Lookup by Position as assistive tasks and propose SATA-MLM and SATA-ELP (see Figure \ref{['fig:method2']}) jailbreaks.
• Figure 2: Overview of the Element Lookup by Position (ELP) assistive task and the SATA-ELP jailbreak.
• Figure 3: ASR comparison of baseline methods (DrAttack and ArtPrompt) vs. SATA-MLM and SATA-ELP across different behavior categories in the JBB dataset. Results for SATA and ArtPrompt are reported under ensemble configuration. Detailed ASR values, including overall ASR, are provided in Appendix \ref{['app:ASR-JBB-table']}, Table \ref{['tab:JBB-result-table']}.
• Figure 4: Cosine similarity across all 32 layers of LLama3-8B. Each similarity value is averaged over 20 selected successful jailbreak examples from the AdvBench dataset.
• Figure 5: Perplexity of each harmful instruction in AdvBench and perplexity of the corresponding adversarial prompt generated by SATA-MLM and SATA-ELP with different masking granularities.