SearchAttack: Red-Teaming LLMs against Real-World Threats via Framing Unsafe Web Information-Seeking Tasks

TL;DR

This work introduces SearchAttack, a dual-stage red-teaming framework that outsources harmful semantics to open web content and uses retrieval-aware rubrics to stress-test how search-augmented LLMs can be steered toward real-world harm. It formalizes an Attack Value framework with fact-checking grounded in external evidence and constructs ShadowRisk, a socio-temporal harm benchmark derived from real-world content. Through extensive experiments across multiple victim models and search configurations, SearchAttack demonstrates high effectiveness and identifies persistent vulnerabilities even under defense prompts, underscoring the need for retrieval-aware safety alignment and robust governance for agentic AI. The study provides a comprehensive methodology for evaluating and hardening search-augmented systems against open-web jailbreaks, with practical implications for model safety, evaluation protocols, and policy making.

Abstract

Recently, people have suffered and become increasingly aware of the unreliability gap in LLMs for open and knowledge-intensive tasks, and thus turn to search-augmented LLMs to mitigate this issue. However, when the search engine is triggered for harmful tasks, the outcome is no longer under the LLM's control. Once the returned content directly contains targeted, ready-to-use harmful takeaways, the LLM's safeguards cannot withdraw that exposure. Motivated by this dilemma, we identify web search as a critical attack surface and propose \textbf{\textit{SearchAttack}} for red-teaming. SearchAttack outsources the harmful semantics to web search, retaining only the query's skeleton and fragmented clues, and further steers LLMs to reconstruct the retrieved content via structural rubrics to achieve malicious goals. Extensive experiments are conducted to red-team the search-augmented LLMs for responsible vulnerability assessment. Empirically, SearchAttack demonstrates strong effectiveness in attacking these systems.

Figures (25)

• Figure 1: Comparison of jailbreak outcomes under different settings. (a) With web search, the model retrieves real-time data to synthesize actionable harm, amplifying the threat. (b) Without web search, the model outputs only outdated, generic content, even when jailbroken.
• Figure 2: Overview of our dual-stage red-teaming method SearchAttack. ❶ Left: The synthesis framework of SearchAttack's attack payloads, which is an agent-collaborative transmuter equipped with web search capabilities. ❷ Right: A concrete case of SearchAttack targeting an LLM with dual-stage payloads and inducing web search for harmful knowledge injection.
• Figure 3: Construction of task-specific rubrics. SearchAttack exploits the reward-chasing behavior in RLVR-trained models by reverse-engineering graded criteria from evaluators. The pursuit of master-level (score 6) objectives silently hacks the model's optimization bias to elicit detailed harmful content, overriding safety alignment.
• Figure 4: A jailbreak case identified by LLM-as-a-Judge evaluators with severe factual errors. This suggests that content-based jailbreak evaluation may overlook the factuality of truly harmful claims, motivating our approach to decouple Attack Value (AtV) evaluation from ASR judgement with a fact-checking framework.
• Figure 5: Attack Value (Scope, Fidelity) comparison across models and search settings on AdvBench.