Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

SOGPTSpotter: Detecting ChatGPT-Generated Answers on Stack Overflow

Suyu Ma, Chunyang Chen, Hourieh Khalajzadeh, John Grundy

TL;DR

This work addresses the challenge of detecting ChatGPT-generated answers on Stack Overflow, where AI-sourced content can undermine information quality. It introduces SOGPTSpotter, a BigBird-based Siamese network trained with triplet loss to distinguish ChatGPT outputs from human and reference answers by leveraging long-context representations and a reference-answer comparison. Through a large triplet dataset of Stack Overflow posts and diverse ChatGPT prompts, the approach achieves state-of-the-art detection performance, robustness to adversarial edits, and strong generalization across domains and other LLMs, with a real-world case study demonstrating practical moderation gains. The results suggest substantial potential for real-time AI-content moderation on Q&A platforms and highlight avenues for extending the method to other domains and future language models.

Abstract

Stack Overflow is a popular Q&A platform where users ask technical questions and receive answers from a community of experts. Recently, there has been a significant increase in the number of answers generated by ChatGPT, which can lead to incorrect and unreliable information being posted on the site. While Stack Overflow has banned such AI-generated content, detecting whether a post is ChatGPT-generated remains a challenging task. We introduce a novel approach, SOGPTSpotter, that employs Siamese Neural Networks, leveraging the BigBird model and the Triplet loss, to detect ChatGPT-generated answers on Stack Overflow. We use triplets of human answers, reference answers, and ChatGPT answers. Our empirical evaluation reveals that our approach outperforms well-established baselines like GPTZero, DetectGPT, GLTR, BERT, RoBERTa, and GPT-2 in identifying ChatGPT-synthesized Stack Overflow responses. We also conducted an ablation study to show the effectiveness of our model. Additional experiments were conducted to assess various factors, including the impact of text length, the model's robustness against adversarial attacks, and its generalization capabilities across different domains and large language models. We also conducted a real-world case study on Stack Overflow. Using our tool's recommendations, Stack Overflow moderators were able to identify and take down ChatGPT-suspected generated answers, demonstrating the practical applicability and effectiveness of our approach.

SOGPTSpotter: Detecting ChatGPT-Generated Answers on Stack Overflow

TL;DR

This work addresses the challenge of detecting ChatGPT-generated answers on Stack Overflow, where AI-sourced content can undermine information quality. It introduces SOGPTSpotter, a BigBird-based Siamese network trained with triplet loss to distinguish ChatGPT outputs from human and reference answers by leveraging long-context representations and a reference-answer comparison. Through a large triplet dataset of Stack Overflow posts and diverse ChatGPT prompts, the approach achieves state-of-the-art detection performance, robustness to adversarial edits, and strong generalization across domains and other LLMs, with a real-world case study demonstrating practical moderation gains. The results suggest substantial potential for real-time AI-content moderation on Q&A platforms and highlight avenues for extending the method to other domains and future language models.

Abstract

Stack Overflow is a popular Q&A platform where users ask technical questions and receive answers from a community of experts. Recently, there has been a significant increase in the number of answers generated by ChatGPT, which can lead to incorrect and unreliable information being posted on the site. While Stack Overflow has banned such AI-generated content, detecting whether a post is ChatGPT-generated remains a challenging task. We introduce a novel approach, SOGPTSpotter, that employs Siamese Neural Networks, leveraging the BigBird model and the Triplet loss, to detect ChatGPT-generated answers on Stack Overflow. We use triplets of human answers, reference answers, and ChatGPT answers. Our empirical evaluation reveals that our approach outperforms well-established baselines like GPTZero, DetectGPT, GLTR, BERT, RoBERTa, and GPT-2 in identifying ChatGPT-synthesized Stack Overflow responses. We also conducted an ablation study to show the effectiveness of our model. Additional experiments were conducted to assess various factors, including the impact of text length, the model's robustness against adversarial attacks, and its generalization capabilities across different domains and large language models. We also conducted a real-world case study on Stack Overflow. Using our tool's recommendations, Stack Overflow moderators were able to identify and take down ChatGPT-suspected generated answers, demonstrating the practical applicability and effectiveness of our approach.
Paper Structure (30 sections, 6 equations, 8 figures, 5 tables)

This paper contains 30 sections, 6 equations, 8 figures, 5 tables.

Table of Contents

  1. Introduction
  2. Background
  3. Stack Overflow
  4. ChatGPT
  5. Siamese Neural Networks
  6. Our Approach
  7. Overview of SOGPTSpotter
  8. Dataset Preparation
  9. Data Selection and Filtering
  10. Generating Reference Answers
  11. Generating ChatGPT Answers
  12. BigBird-Based Siamese Network with Triplet Loss
  13. Implementation
  14. Evaluation Study Design
  15. Research Questions
  16. ...and 15 more sections

Figures (8)

  • Figure 1: Example of the misleading ChatGPT answer
  • Figure 2: Workflow of Our Approach
  • Figure 3: SQL Query for Selecting High-Quality Post Questions and Answers
  • Figure 4: Structure of the BigBird-based Siamese Network
  • Figure 5: Attention Mechanisms in BigBird
  • ...and 3 more figures