Analysis and prevention of AI-based phishing email attacks
Chibuike Samuel Eze, Lior Shamir
TL;DR
This work addresses the rising threat of AI-generated phishing emails by creating a public corpus of 865 AI-generated messages and evaluating multiple text-analysis methods to distinguish them from human-authored and traditional phishing emails. It employs MALLET-based classifiers, a style-descriptor approach (UDAT), and a lightweight LSTM, plus an ensemble to maximize detection performance. All methods achieve high accuracy, with MALLET-Naive Bayes at 99.3% and UDAT at 98%, and ensemble performance around 99.5%, demonstrating that AI-generated phishing can be effectively identified when trained on AI-generated data. The public corpus and demonstrated methodology provide baselines for future defenses against generative-AI-powered phishing and highlight the importance of including AI-generated data in training regimes.
Abstract
Phishing email attacks are among the most common and most harmful cybersecurity attacks. With the emergence of generative AI, phishing attacks can be based on emails generated automatically, making it more difficult to detect them. That is, instead of a single email format sent to a large number of recipients, generative AI can be used to send each potential victim a different email, making it more difficult for cybersecurity systems to identify the scam email before it reaches the recipient. Here we describe a corpus of AI-generated phishing emails. We also use different machine learning tools to test the ability of automatic text analysis to identify AI-generated phishing emails. The results are encouraging, and show that machine learning tools can identify an AI-generated phishing email with high accuracy compared to regular emails or human-generated scam email. By applying descriptive analytic, the specific differences between AI-generated emails and manually crafted scam emails are profiled, and show that AI-generated emails are different in their style from human-generated phishing email scams. Therefore, automatic identification tools can be used as a warning for the user. The paper also describes the corpus of AI-generated phishing emails that is made open to the public, and can be used for consequent studies. While the ability of machine learning to detect AI-generated phishing email is encouraging, AI-generated phishing emails are different from regular phishing emails, and therefore it is important to train machine learning systems also with AI-generated emails in order to repel future phishing attacks that are powered by generative AI.