The Pitfalls of Publishing in the Age of LLMs: Strange and Surprising Adventures with a High-Impact NLP Journal

TL;DR

The paper addresses a pressing problem: how LLMs can disrupt scholarly publishing by enabling AI-generated contributions in peer review and raising confidentiality concerns. It presents a case study of a deception-detection manuscript submitted to a top NLP/CL journal where a reviewer’s feedback appears machine-written. The authors document editor responses, the author’s correspondence, and the subsequent suggestion to build an ethics-policy for misconduct in peer review. They also experiment with ChatGPT-4 to assess the suspicious review, finding that the model often cannot flag AI-generated content or hidden ethical violations, highlighting a gap in AI-assisted quality control. Overall, the work calls for robust editorial policies, better reviewer screening, and transparency about AI tools to protect authors and the integrity of the publication process.

Abstract

We show the fraught side of the academic publishing realm and illustrate it through a recent case study with an NLP journal.