Demystifying RCE Vulnerabilities in LLM-Integrated Apps
Tong Liu, Zizhuang Deng, Guozhu Meng, Yuekang Li, Kai Chen
TL;DR
The paper tackles the security risk of remote code execution (RCE) in LLM-integrated apps by introducing LLMSmith, a framework that combines lightweight static analysis, targeted app collection, and a prompt-based exploitation workflow. It demonstrates the discovery of 20 vulnerabilities (13 CVEs) across 11 frameworks and successful exploitation in 17 apps, accompanied by a hazard analysis for app hosts and benign users. The study provides a detailed methodology for vulnerability detection, exploitation, and real-world impact assessment, including defensive mitigations such as permission management, environment isolation, and prompt-analysis defenses. The findings highlight practical security risks in the evolving LLM ecosystem and offer guidance for developers and vendors to bolster defenses while acknowledging the need for broader language coverage and vulnerability types in future work.
Abstract
LLMs show promise in transforming software development, with a growing interest in integrating them into more intelligent apps. Frameworks like LangChain aid LLM-integrated app development, offering code execution utility/APIs for custom actions. However, these capabilities theoretically introduce Remote Code Execution (RCE) vulnerabilities, enabling remote code execution through prompt injections. No prior research systematically investigates these frameworks' RCE vulnerabilities or their impact on applications and exploitation consequences. Therefore, there is a huge research gap in this field. In this study, we propose LLMSmith to detect, validate and exploit the RCE vulnerabilities in LLM-integrated frameworks and apps. To achieve this goal, we develop two novel techniques, including 1) a lightweight static analysis to examine LLM integration mechanisms, and construct call chains to identify RCE vulnerabilities in frameworks; 2) a systematical prompt-based exploitation method to verify and exploit the found vulnerabilities in LLM-integrated apps. This technique involves various strategies to control LLM outputs, trigger RCE vulnerabilities and launch subsequent attacks. Our research has uncovered a total of 20 vulnerabilities in 11 LLM-integrated frameworks, comprising 19 RCE vulnerabilities and 1 arbitrary file read/write vulnerability. Of these, 17 have been confirmed by the framework developers, with 11 vulnerabilities being assigned CVE IDs. For the 51 apps potentially affected by RCE, we successfully executed attacks on 17 apps, 16 of which are vulnerable to RCE and 1 to SQL injection. Furthermore, we conduct a comprehensive analysis of these vulnerabilities and construct practical attacks to demonstrate the hazards in reality. Last, we propose several mitigation measures for both framework and app developers to counteract such attacks.