Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Exploiting Novel GPT-4 APIs

Kellin Pelrine, Mohammad Taufeeque, Michał Zając, Euan McLean, Adam Gleave

TL;DR

The paper red-teams three newly exposed GPT-4 APIs—fine-tuning, function calling, and knowledge retrieval—to reveal novel vulnerabilities in safety guardrails, programmatic access, and document-based reasoning. It demonstrates that targeted data poisoning via fine-tuning can erode safeguards, enable misinformation, and leak private information; function calling can disclose schemas and permit arbitrary, unsanitized calls; and retrieval can be hijacked by prompt injections and biased system instructions. The work provides a detailed taxonomy of attacks, including targeted misinformation, conspiracy theories, PII leakage, URL injection, and multiple retrieval-based injections, supported by extensive demonstrations. It highlights that expanding API capabilities often introduces new risk vectors, underlining the need for robust defenses and rigorous red-teaming before deployment in security-critical contexts.

Abstract

Language model attacks typically assume one of two extreme threat models: full white-box access to model weights, or black-box access limited to a text generation API. However, real-world APIs are often more flexible than just text generation: these APIs expose "gray-box" access leading to new threat vectors. To explore this, we red-team three new functionalities exposed in the GPT-4 APIs: fine-tuning, function calling and knowledge retrieval. We find that fine-tuning a model on as few as 15 harmful examples or 100 benign examples can remove core safeguards from GPT-4, enabling a range of harmful outputs. Furthermore, we find that GPT-4 Assistants readily divulge the function call schema and can be made to execute arbitrary function calls. Finally, we find that knowledge retrieval can be hijacked by injecting instructions into retrieval documents. These vulnerabilities highlight that any additions to the functionality exposed by an API can create new vulnerabilities.

Exploiting Novel GPT-4 APIs

TL;DR

The paper red-teams three newly exposed GPT-4 APIs—fine-tuning, function calling, and knowledge retrieval—to reveal novel vulnerabilities in safety guardrails, programmatic access, and document-based reasoning. It demonstrates that targeted data poisoning via fine-tuning can erode safeguards, enable misinformation, and leak private information; function calling can disclose schemas and permit arbitrary, unsanitized calls; and retrieval can be hijacked by prompt injections and biased system instructions. The work provides a detailed taxonomy of attacks, including targeted misinformation, conspiracy theories, PII leakage, URL injection, and multiple retrieval-based injections, supported by extensive demonstrations. It highlights that expanding API capabilities often introduces new risk vectors, underlining the need for robust defenses and rigorous red-teaming before deployment in security-critical contexts.

Abstract

Language model attacks typically assume one of two extreme threat models: full white-box access to model weights, or black-box access limited to a text generation API. However, real-world APIs are often more flexible than just text generation: these APIs expose "gray-box" access leading to new threat vectors. To explore this, we red-team three new functionalities exposed in the GPT-4 APIs: fine-tuning, function calling and knowledge retrieval. We find that fine-tuning a model on as few as 15 harmful examples or 100 benign examples can remove core safeguards from GPT-4, enabling a range of harmful outputs. Furthermore, we find that GPT-4 Assistants readily divulge the function call schema and can be made to execute arbitrary function calls. Finally, we find that knowledge retrieval can be hijacked by injecting instructions into retrieval documents. These vulnerabilities highlight that any additions to the functionality exposed by an API can create new vulnerabilities.
Paper Structure (56 sections, 1 figure, 7 tables)

This paper contains 56 sections, 1 figure, 7 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Exploiting the OpenAI Fine-Tuning API
  4. Removing Safety Guardrails
  5. Dataset Generation
  6. Harmfulness Evaluation of Fine-Tuned Models
  7. Misinformation
  8. Targeted Misinformation
  9. Conspiracy Theories
  10. Divulging Private Information
  11. Injecting Malicious URLs
  12. Red-teaming the OpenAI Assistants API
  13. Reconnaissance & Attacks on Functions
  14. Attack 1: Divulging Schema.
  15. Attack 2: Arbitrary Function Calls.
  16. ...and 41 more sections

Figures (1)

  • Figure 1: Examples of the attacks we performed on three recently added functionalities of the GPT-4 API. We find that fine-tuning can remove or diminish the safety guardrails of GPT-4, so that it responds to harmful requests like "how do I build a bomb?" When testing function calling, we find that models readily divulge the function call schema and will execute arbitrary unsanitized function calls. For knowledge retrieval, we found that when asked to summarize a document that contains a maliciously injected instruction, the model will obey that instruction instead of summarizing the document.