Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Safeguarding Crowdsourcing Surveys from ChatGPT with Prompt Injection

Chaofan Wang, Samuel Kernan Freire, Mo Zhang, Jing Wei, Jorge Goncalves, Vassilis Kostakos, Zhanna Sarsenbayeva, Christina Schneegass, Alessandro Bozzon, Evangelos Niforatos

TL;DR

The paper tackles the risk of large language models automatically answering crowdsourcing surveys and proposes a prompt-injection framework to detect such responses. It formalizes the PROMPTINJECT approach, analyzes how question type, prompt construction, and injection position affect effectiveness, and validates the method through a four-context daily-life case study. The results show high injection effectiveness across configurations, supporting the feasibility of using prompt injection as a quality-assurance tool in survey design, complemented by an open-source software tool. The work highlights practical implications for maintaining data integrity in crowdsourcing and discusses alternative defenses, limitations, and directions for future research.

Abstract

ChatGPT and other large language models (LLMs) have proven useful in crowdsourcing tasks, where they can effectively annotate machine learning training data. However, this means that they also have the potential for misuse, specifically to automatically answer surveys. LLMs can potentially circumvent quality assurance measures, thereby threatening the integrity of methodologies that rely on crowdsourcing surveys. In this paper, we propose a mechanism to detect LLM-generated responses to surveys. The mechanism uses "prompt injection", such as directions that can mislead LLMs into giving predictable responses. We evaluate our technique against a range of question scenarios, types, and positions, and find that it can reliably detect LLM-generated responses with more than 93% effectiveness. We also provide an open-source software to help survey designers use our technique to detect LLM responses. Our work is a step in ensuring that survey methodologies remain rigorous vis-a-vis LLMs.

Safeguarding Crowdsourcing Surveys from ChatGPT with Prompt Injection

TL;DR

The paper tackles the risk of large language models automatically answering crowdsourcing surveys and proposes a prompt-injection framework to detect such responses. It formalizes the PROMPTINJECT approach, analyzes how question type, prompt construction, and injection position affect effectiveness, and validates the method through a four-context daily-life case study. The results show high injection effectiveness across configurations, supporting the feasibility of using prompt injection as a quality-assurance tool in survey design, complemented by an open-source software tool. The work highlights practical implications for maintaining data integrity in crowdsourcing and discusses alternative defenses, limitations, and directions for future research.

Abstract

ChatGPT and other large language models (LLMs) have proven useful in crowdsourcing tasks, where they can effectively annotate machine learning training data. However, this means that they also have the potential for misuse, specifically to automatically answer surveys. LLMs can potentially circumvent quality assurance measures, thereby threatening the integrity of methodologies that rely on crowdsourcing surveys. In this paper, we propose a mechanism to detect LLM-generated responses to surveys. The mechanism uses "prompt injection", such as directions that can mislead LLMs into giving predictable responses. We evaluate our technique against a range of question scenarios, types, and positions, and find that it can reliably detect LLM-generated responses with more than 93% effectiveness. We also provide an open-source software to help survey designers use our technique to detect LLM responses. Our work is a step in ensuring that survey methodologies remain rigorous vis-a-vis LLMs.
Paper Structure (28 sections, 5 figures, 7 tables, 1 algorithm)

This paper contains 28 sections, 5 figures, 7 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Related Work
  3. Crowdsourcing Surveys
  4. Large Language Models
  5. Impact of Large Language Models on Crowdsourcing Studies
  6. Prompt Injection as a Remedy
  7. Prompt Injection in Large Language Models
  8. Prompt Structure
  9. Injecting Attack Prompts
  10. Key Aspects of Attack Prompt Construction and Injection
  11. Construction -- Question Type
  12. Construction -- Method
  13. Injection -- Position
  14. Case Study -- Daily-life Scenario Recommendation
  15. Task Description
  16. ...and 13 more sections

Figures (5)

  • Figure 1: Our adaptation of the PROMPTINJECT framework perez2022ignore specifically for goal hijacking on crowdsourcing platforms.
  • Figure 2: Length of attack prompts across construction methods.
  • Figure 3: Software interface of prep steps for generating an attack.
  • Figure 4: Software interface of manual attack prompt construction.
  • Figure 5: Software interface of automated attack prompt construction.