Clouding the Mirror: Stealthy Prompt Injection Attacks Targeting LLM-based Phishing Detection

TL;DR

This work identifies and systematizes prompt injection risks in multimodal LLM-based phishing detectors through a two-axis taxonomy that couples Attack Techniques with Attack Surfaces. It demonstrates that even advanced models like GPT-5 remain vulnerable to cleverly embedded instructions across HTML, URLs, and visuals, potentially degrading phishing judgments or disrupting pipelines. The authors introduce InjectDefuser, a defense framework combining prompt hardening, allowlist-based retrieval augmentation, and output validation, and show substantial reductions in attack success rates across multiple models and surfaces. The findings offer actionable mitigation strategies and a structured threat model to strengthen future, real-world deployment of LLM-based phishing countermeasures.

Abstract

Phishing sites continue to grow in volume and sophistication. Recent work leverages large language models (LLMs) to analyze URLs, HTML, and rendered content to decide whether a website is a phishing site. While these approaches are promising, LLMs are inherently vulnerable to prompt injection (PI). Because attackers can fully control various elements of phishing sites, this creates the potential for PI that exploits the perceptual asymmetry between LLMs and humans: instructions imperceptible to end users can still be parsed by the LLM and can stealthily manipulate its judgment. The specific risks of PI in phishing detection and effective mitigation strategies remain largely unexplored. This paper presents the first comprehensive evaluation of PI against multimodal LLM-based phishing detection. We introduce a two-dimensional taxonomy, defined by Attack Techniques and Attack Surfaces, that captures realistic PI strategies. Using this taxonomy, we implement diverse attacks and empirically study several representative LLM-based detection systems. The results show that phishing detection with state-of-the-art models such as GPT-5 remains vulnerable to PI. We then propose InjectDefuser, a defense framework that combines prompt hardening, allowlist-based retrieval augmentation, and output validation. Across multiple models, InjectDefuser significantly reduces attack success rates. Our findings clarify the PI risk landscape and offer practical defenses that improve the reliability of next-generation phishing countermeasures.

Figures (8)

• Figure 1: Prompt injection attacks against LLM-based phishing detection exploiting perceptual asymmetry. Attackers inject hidden instructions into attacker-controlled web content that victims cannot perceive but that the LLM ingests (e.g., via HTML, screenshots, or URLs), potentially leading to misclassification of phishing attempts or disruption of the pipeline.
• Figure 2: Example of hiding prompt injection messages by inserting multiple spaces ( ) into the title tag. Although nothing appears after the original "iCloud" title in the browser tab, the HTML shows that the PI message is included.
• Figure 3: Example of PI embedded in HTML Visible Content. Although it is nearly invisible to users, the presence of the text becomes apparent when it is selected and highlighted, and LLMs can read this text.
• Figure 4: Example of PI placed at page bottom via embedded resources using small fonts.
• Figure 5: Overview of InjectDefuser.