Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Detection of ChatGPT Fake Science with the xFakeSci Learning Algorithm

Ahmed Abdeen Hamed, Xindong Wu

TL;DR

The study addresses the rising challenge of AI-generated fake science by proposing xFakeSci, a network-driven classifier that distinguishes ChatGPT-generated biomedical abstracts from authentic PubMed abstracts. It leverages prompt-engineering to produce a large, labeled corpus across Alzheimer's, cancer, and depression, and builds two TF-IDF bigram networks (PubMed vs GPT) whose structure and bigram ratios reveal detectable differences. Calibration using data-driven ratio ranges and a proximity heuristic enables robust, multi-source label prediction, achieving F1 scores of 80–94% and outperforming classical baselines (38–52%). This work provides a practical forensic tool for safeguarding scientific integrity in the era of generative AI and points to future enhancements via knowledge graphs and additional data sources to broaden detection coverage.

Abstract

Generative AI tools exemplified by ChatGPT are becoming a new reality. This study is motivated by the premise that ``AI generated content may exhibit a distinctive behavior that can be separated from scientific articles''. In this study, we show how articles can be generated using means of prompt engineering for various diseases and conditions. We then show how we tested this premise in two phases and prove its validity. Subsequently, we introduce xFakeSci, a novel learning algorithm, that is capable of distinguishing ChatGPT-generated articles from publications produced by scientists. The algorithm is trained using network models driven from both sources. As for the classification step, it was performed using 300 articles per condition. The actual label steps took place against an equal mix of 50 generated articles and 50 authentic PubMed abstracts. The testing also spanned publication periods from 2010 to 2024 and encompassed research on three distinct diseases: cancer, depression, and Alzheimer's. Further, we evaluated the accuracy of the xFakeSci algorithm against some of the classical data mining algorithms (e.g., Support Vector Machines, Regression, and Naive Bayes). The xFakeSci algorithm achieved F1 scores ranging from 80% to 94%, outperforming common data mining algorithms, which scored F1 values between 38% and 52%. We attribute the noticeable difference to the introduction of calibration and a proximity distance heuristic, which underscores this promising performance. Indeed, the prediction of fake science generated by ChatGPT presents a considerable challenge. Nonetheless, the introduction of the xFakeSci algorithm is a significant step on the way to combating fake science.

Detection of ChatGPT Fake Science with the xFakeSci Learning Algorithm

TL;DR

The study addresses the rising challenge of AI-generated fake science by proposing xFakeSci, a network-driven classifier that distinguishes ChatGPT-generated biomedical abstracts from authentic PubMed abstracts. It leverages prompt-engineering to produce a large, labeled corpus across Alzheimer's, cancer, and depression, and builds two TF-IDF bigram networks (PubMed vs GPT) whose structure and bigram ratios reveal detectable differences. Calibration using data-driven ratio ranges and a proximity heuristic enables robust, multi-source label prediction, achieving F1 scores of 80–94% and outperforming classical baselines (38–52%). This work provides a practical forensic tool for safeguarding scientific integrity in the era of generative AI and points to future enhancements via knowledge graphs and additional data sources to broaden detection coverage.

Abstract

Generative AI tools exemplified by ChatGPT are becoming a new reality. This study is motivated by the premise that ``AI generated content may exhibit a distinctive behavior that can be separated from scientific articles''. In this study, we show how articles can be generated using means of prompt engineering for various diseases and conditions. We then show how we tested this premise in two phases and prove its validity. Subsequently, we introduce xFakeSci, a novel learning algorithm, that is capable of distinguishing ChatGPT-generated articles from publications produced by scientists. The algorithm is trained using network models driven from both sources. As for the classification step, it was performed using 300 articles per condition. The actual label steps took place against an equal mix of 50 generated articles and 50 authentic PubMed abstracts. The testing also spanned publication periods from 2010 to 2024 and encompassed research on three distinct diseases: cancer, depression, and Alzheimer's. Further, we evaluated the accuracy of the xFakeSci algorithm against some of the classical data mining algorithms (e.g., Support Vector Machines, Regression, and Naive Bayes). The xFakeSci algorithm achieved F1 scores ranging from 80% to 94%, outperforming common data mining algorithms, which scored F1 values between 38% and 52%. We attribute the noticeable difference to the introduction of calibration and a proximity distance heuristic, which underscores this promising performance. Indeed, the prediction of fake science generated by ChatGPT presents a considerable challenge. Nonetheless, the introduction of the xFakeSci algorithm is a significant step on the way to combating fake science.
Paper Structure (19 sections, 6 equations, 6 figures, 8 tables, 5 algorithms)

This paper contains 19 sections, 6 equations, 6 figures, 8 tables, 5 algorithms.

Table of Contents

  1. Highlights
  2. Introduction
  3. Literature Background
  4. Results
  5. Outcome of Evaluating the Premise
  6. Phase I: Analysis of Topological Properties of Network Training Models
  7. Phase II: Further Testing the Distinctive Behavior in ChatGPT-Generated Documents
  8. Outcome of Label Prediction of Multi-Mode Classification Experiments
  9. Outcome of Publication Period as a Factor for the Multi-Mode Classification Experiments
  10. Outcome of xFakeSci Performance Analysis against other Data Mining Algorithms
  11. Methods
  12. Data Collection
  13. ChatGPT Prompt Engineering for Article Generation
  14. Prediction of Fake Articles using xFakeSci
  15. Model Derivation and Network Construction
  16. ...and 4 more sections

Figures (6)

  • Figure 1: Nodes to Edges Ratios for Different Datasets ChatGPT vs Scientific Articles
  • Figure 2: Comparison of Calibrating Ratio Means for the Cancer Disease.
  • Figure 3: Comparison of Calibrating Ratio Means for the Depression Disease.
  • Figure 4: Multi-Mode Experiments: F1 Classification Scores for (Cancer, Alzheimer's, and Depression) for publications gathered in period (2020-2024.)
  • Figure 5: Showing (TP, TN, FP, FN) Mertics and F1 Scores for xFakeSci vs Classical Data Mining algorithms Using Cancer Dataset From 2020-2024
  • ...and 1 more figures