Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

GPT in Sheep's Clothing: The Risk of Customized GPTs

Sagiv Antebi, Noam Azulay, Edan Habler, Ben Ganon, Asaf Shabtai, Yuval Elovici

TL;DR

The paper investigates security risks posed by OpenAI's customizable GPTs, arguing that attackers can abuse these shared AI agents to conduct privacy- and security-threatening activities. It introduces a threat taxonomy spanning Vulnerability Steering, Malicious Injection, and Information Theft, and demonstrates concrete attack scenarios, including $N$-Day exploit guidance, insecure code generation, malicious libraries, and phishing via GPTs. The authors propose practical mitigations—self-checks, verification, community reputation, explicit link presentation, and API-call scrutiny—showing these defenses can detect or deter many attacks, though they acknowledge the need for more robust, systemic safeguards. The work highlights the importance of cautious use, transparent builder authentication, and community mechanisms to protect users as GPT customization evolves.

Abstract

In November 2023, OpenAI introduced a new service allowing users to create custom versions of ChatGPT (GPTs) by using specific instructions and knowledge to guide the model's behavior. We aim to raise awareness of the fact that GPTs can be used maliciously, posing privacy and security risks to their users.

GPT in Sheep's Clothing: The Risk of Customized GPTs

TL;DR

The paper investigates security risks posed by OpenAI's customizable GPTs, arguing that attackers can abuse these shared AI agents to conduct privacy- and security-threatening activities. It introduces a threat taxonomy spanning Vulnerability Steering, Malicious Injection, and Information Theft, and demonstrates concrete attack scenarios, including -Day exploit guidance, insecure code generation, malicious libraries, and phishing via GPTs. The authors propose practical mitigations—self-checks, verification, community reputation, explicit link presentation, and API-call scrutiny—showing these defenses can detect or deter many attacks, though they acknowledge the need for more robust, systemic safeguards. The work highlights the importance of cautious use, transparent builder authentication, and community mechanisms to protect users as GPT customization evolves.

Abstract

In November 2023, OpenAI introduced a new service allowing users to create custom versions of ChatGPT (GPTs) by using specific instructions and knowledge to guide the model's behavior. We aim to raise awareness of the fact that GPTs can be used maliciously, posing privacy and security risks to their users.
Paper Structure (22 sections, 8 figures)

This paper contains 22 sections, 8 figures.

Table of Contents

  1. Introduction
  2. Creating GPTs
  3. GPTs Threat Taxonomy
  4. Vulnerability Steering
  5. Malicious Injection
  6. Information Theft
  7. Vulnerability Steering
  8. N-Day Exploit Attacks
  9. Insecure Practices
  10. Malicious Injection
  11. Malicious Code Snippet
  12. Malicious Library
  13. Information Theft
  14. Direct Phishing
  15. Third-Party Phishing
  16. ...and 7 more sections

Figures (8)

  • Figure 1: GPTs Threat Taxonomy
  • Figure 2: N-Day Exploit Attack - In the Log4Shell attack https://chat.openai.com/share/a0389492-3a3c-4175-888d-c53f32d17d3f - the GPT makes recommendations and guides the user as to how to change their Java version to an earlier (vulnerable) one. Then the GPT injects the vulnerable code snippet into the code provided.
  • Figure 3: SQL Injection https://chat.openai.com/share/c2607d59-c0eb-4c91-8aea-96beb78a2337 - The GPT provides the vulnerable SQL injection script.
  • Figure 4: Buffer Overflow https://chat.openai.com/share/cfa55e0f-201e-46d9-a8fa-b795f9ad8ca2 - The GPT adds the script to the code, although it makes no sense to put it there, exposing the program to a buffer overflow attack.
  • Figure 5: Malicious Code Snippet https://chat.openai.com/share/0dcabcc4-c4b1-4060-929f-8bc50aceb182- The GPT provides the code, embedded with malicious elements that appear to be normal code.
  • ...and 3 more figures