Synthetic Cancer -- Augmenting Worms with LLMs
Benjamin Zimmerman, David Zollikofer
TL;DR
This paper identifies a cybersecurity threat in which AI-enabled malware uses LLMs to autonomously rewrite its code to evade signature-based defenses and to craft phishing-like email replies for targeted propagation. It outlines a three-stage attack—initial infection via email, self-redaction by an LLM to produce diverse variants, and propagation through socially engineered responses—and discusses a minimal prototype. The authors acknowledge practical defense challenges, arguing that relying solely on detectors or user training is insufficient and call for broader defense research. Overall, the work highlights the critical risk of AI-augmented malware and the need for cross-cutting security strategies beyond traditional signature-based approaches.
Abstract
With increasingly sophisticated large language models (LLMs), the potential for abuse rises drastically. As a submission to the Swiss AI Safety Prize, we present a novel type of metamorphic malware leveraging LLMs for two key processes. First, LLMs are used for automatic code rewriting to evade signature-based detection by antimalware programs. The malware then spreads its copies via email by utilizing an LLM to socially engineer email replies to encourage recipients to execute the attached malware. Our submission includes a functional minimal prototype, highlighting the risks that LLMs pose for cybersecurity and underscoring the need for further research into intelligent malware.