Robust ML-based Detection of Conventional, LLM-Generated, and Adversarial Phishing Emails Using Advanced Text Preprocessing

TL;DR

This work tackles phishing detection in the era of adversarial manipulation and AI-generated content by proposing a robust ML-based system that integrates advanced text preprocessing (spelling correction and word splitting) with multiple NLP feature extractors (TF-IDF, Word2Vec, GloVe) and traditional classifiers. It demonstrates that the enhanced preprocessing consistently improves performance in both development and deployment settings and strengthens resilience against adversarial perturbations and LLM-generated phishing emails. Through extensive evaluation on public datasets and TextAttack-based attacks, the approach shows substantial robustness gains and practical viability for real-world email security. The results suggest actionable directions for future work, including header-based features, fine-tuning modern LLMs, multi-class phishing categorization, and explainable AI methods to increase user trust and system transparency.

Abstract

Phishing remains a critical cybersecurity threat, especially with the advent of large language models (LLMs) capable of generating highly convincing malicious content. Unlike earlier phishing attempts which are identifiable by grammatical errors, misspellings, incorrect phrasing, and inconsistent formatting, LLM generated emails are grammatically sound, contextually relevant, and linguistically natural. These advancements make phishing emails increasingly difficult to distinguish from legitimate ones, challenging traditional detection mechanisms. Conventional phishing detection systems often fail when faced with emails crafted by LLMs or manipulated using adversarial perturbation techniques. To address this challenge, we propose a robust phishing email detection system featuring an enhanced text preprocessing pipeline. This pipeline includes spelling correction and word splitting to counteract adversarial modifications and improve detection accuracy. Our approach integrates widely adopted natural language processing (NLP) feature extraction techniques and machine learning algorithms. We evaluate our models on publicly available datasets comprising both phishing and legitimate emails, achieving a detection accuracy of 94.26% and F1-score of 84.39% in model deployment setting. To assess robustness, we further evaluate our models using adversarial phishing samples generated by four attack methods in Python TextAttack framework. Additionally, we evaluate models' performance against phishing emails generated by LLMs including ChatGPT and Llama. Results highlight the resilience of models against evolving AI-powered phishing threats.

Figures (4)

• Figure 1: Level 1 and 2 text preprocessing of emails
• Figure 2: ROC curves for models trained with Baseline dataset (Level 1 preprocessing) and Advanced dataset (Level 1 and 2 preprocessing)
• Figure 3: Evaluation of adversarial robustness of phishing detection systems
• Figure 4: Comparison of post-attack accuracy for models trained with Word2Vec embedding