AbuseGPT: Abuse of Generative AI ChatBots to Create Smishing Campaigns

TL;DR

The paper investigates how contemporary generative AI chatbots can be exploited to design smishing campaigns. By formulating the AbuseGPT methodology and posing targeted research questions, the authors demonstrate that jailbreak prompts can bypass safeguards and elicit smishing themes, example messages, toolkits, and even fake URLs from models like ChatGPT. The study discusses cross chatbot defenses, the evolving nature of AI driven social engineering, and advocates multi layer defenses including awareness, education, and URL verification. While revealing significant risks, it also outlines limitations such as time sensitive prompt evasion and the absence of human subject testing, and proposes directions for defense oriented research and policy safeguards.

Abstract

SMS phishing, also known as "smishing", is a growing threat that tricks users into disclosing private information or clicking into URLs with malicious content through fraudulent mobile text messages. In recent past, we have also observed a rapid advancement of conversational generative AI chatbot services (e.g., OpenAI's ChatGPT, Google's BARD), which are powered by pre-trained large language models (LLMs). These AI chatbots certainly have a lot of utilities but it is not systematically understood how they can play a role in creating threats and attacks. In this paper, we propose AbuseGPT method to show how the existing generative AI-based chatbot services can be exploited by attackers in real world to create smishing texts and eventually lead to craftier smishing campaigns. To the best of our knowledge, there is no pre-existing work that evidently shows the impacts of these generative text-based models on creating SMS phishing. Thus, we believe this study is the first of its kind to shed light on this emerging cybersecurity threat. We have found strong empirical evidences to show that attackers can exploit ethical standards in the existing generative AI-based chatbot services by crafting prompt injection attacks to create newer smishing campaigns. We also discuss some future research directions and guidelines to protect the abuse of generative AI-based services and safeguard users from smishing attacks.

Figures (12)

• Figure 1: Overview of proposed AbuseGPT method
• Figure 2: Asking ChatGPT directly to give an SMS phishing message without jailbreaking
• Figure 3: Asking indirectly for a smishing message without jailbreaking
• Figure 4: 'AIM' Jailbreak prompt
• Figure 5: Unethical response after jailbreaking to design effective smishing