Security Analysis of ChatGPT: Threats and Privacy Risks
Yushan Xiang, Zhongwen Li, Xiaoqi Li
TL;DR
This paper surveys the security threats and privacy risks facing ChatGPT, distinguishing traditional social-engineering risks from emerging, LLM-specific threats such as false information, data poisoning, sponge samples, and privacy breaches. It analyzes the underlying technical foundations (Transformer and self-attention) and demonstrates attack/defense scenarios, including model theft, session reconfiguration, and membership inference. The authors also discuss ethical and legal challenges and illustrate practical security practices where ChatGPT could both enable attacks (via code and phishing content) and bolster defenses (via WAF rules and vulnerability analysis). They propose concrete mitigations, including vulnerability monitoring, stronger privacy protections, improved input filtering, and better data quality supervision to guide responsible deployment.
Abstract
As artificial intelligence technology continues to advance, chatbots are becoming increasingly powerful. Among them, ChatGPT, launched by OpenAI, has garnered widespread attention globally due to its powerful natural language processing capabilities based on the GPT model, which enables it to engage in natural conversations with users, understand various forms of linguistic expressions, and generate useful information and suggestions. However, as its application scope expands, user demand grows, and malicious attacks related to it become increasingly frequent, the security threats and privacy risks faced by ChatGPT are gradually coming to the forefront. In this paper, the security of ChatGPT is mainly studied from two aspects, security threats and privacy risks. The article systematically analyzes various types of vulnerabilities involved in the above two types of problems and their causes. Briefly, we discuss the controversies that ChatGPT may cause at the ethical and moral levels. In addition, this paper reproduces several network attack and defense test scenarios by simulating the attacker's perspective and methodology. Simultaneously, it explores the feasibility of using ChatGPT for security vulnerability detection and security tool generation from the defender's perspective.