Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Generating Phishing Attacks using ChatGPT

Sayak Saha Roy, Krishna Vamsi Naragam, Shirin Nilizadeh

TL;DR

This paper investigates how ChatGPT can be exploited to generate phishing websites, including evasive variants, by engineering prompts that bypass safeguards. It decomposes prompt design into design, credential theft, exploit, and data transfer components to assemble functional attack code. The authors demonstrate multiple attack types (regular and evasive) targeting 50 brands, and validate feasibility by hosting on free services with minimal prompts. The work highlights the rapid deployment risk posed by LLMs and underscores the need for robust defenses and policy responses.

Abstract

The ability of ChatGPT to generate human-like responses and understand context has made it a popular tool for conversational agents, content creation, data analysis, and research and innovation. However, its effectiveness and ease of accessibility makes it a prime target for generating malicious content, such as phishing attacks, that can put users at risk. In this work, we identify several malicious prompts that can be provided to ChatGPT to generate functional phishing websites. Through an iterative approach, we find that these phishing websites can be made to imitate popular brands and emulate several evasive tactics that have been known to avoid detection by anti-phishing entities. These attacks can be generated using vanilla ChatGPT without the need of any prior adversarial exploits (jailbreaking).

Generating Phishing Attacks using ChatGPT

TL;DR

This paper investigates how ChatGPT can be exploited to generate phishing websites, including evasive variants, by engineering prompts that bypass safeguards. It decomposes prompt design into design, credential theft, exploit, and data transfer components to assemble functional attack code. The authors demonstrate multiple attack types (regular and evasive) targeting 50 brands, and validate feasibility by hosting on free services with minimal prompts. The work highlights the rapid deployment risk posed by LLMs and underscores the need for robust defenses and policy responses.

Abstract

The ability of ChatGPT to generate human-like responses and understand context has made it a popular tool for conversational agents, content creation, data analysis, and research and innovation. However, its effectiveness and ease of accessibility makes it a prime target for generating malicious content, such as phishing attacks, that can put users at risk. In this work, we identify several malicious prompts that can be provided to ChatGPT to generate functional phishing websites. Through an iterative approach, we find that these phishing websites can be made to imitate popular brands and emulate several evasive tactics that have been known to avoid detection by anti-phishing entities. These attacks can be generated using vanilla ChatGPT without the need of any prior adversarial exploits (jailbreaking).
Paper Structure (17 sections, 7 figures, 1 table)

This paper contains 17 sections, 7 figures, 1 table.

Table of Contents

  1. Introduction
  2. Threat Model
  3. Ethical Considerations
  4. Methodology
  5. Constructing the attacks
  6. Generation of phishing websites
  7. Regular phishing attack
  8. ReCAPTCHA attacks
  9. QR Code attacks
  10. Browser-in-the-Browser attack
  11. iFrame injection/Clickjacking
  12. Exploiting DOM classifiers
  13. Polymorphic URL
  14. Text encoding exploit
  15. Browser fingerprinting attacks
  16. ...and 2 more sections

Figures (7)

  • Figure 1: Breaking down the prompt into functional objects to trick ChatGPT into generating the attack
  • Figure 2: Example of two simple prompts which asks ChatGPT to create a phishing attack both directly (a) and indirectly (b). In both cases, ChatGPT refuses both requests, citing violation of OpenAI TOS. Our goal to engineer prompts that can evade this refusal and lead to functional phishing attacks.
  • Figure 3: Intial landing page generated by ChatGPT which contains a QR code created automatically using QRServer API. Scanning the QR code leads to a different Amazon phishing page (Also designed by ChatGPT).
  • Figure 4: An example of a Browser in the Browser attack generated by ChatGPT. Here clicking on the 'Login with Amazon' button leads to the rogue popup imitating the design and URL of the real Amazon login page.
  • Figure 5: Example of a clickjacking attack generated by ChatGPT which leads to an Amazon phishing page
  • ...and 2 more figures