Usefulness of LLMs as an Author Checklist Assistant for Scientific Papers: NeurIPS'24 Experiment

TL;DR

Qualitative evidence suggests that the LLM contributed to improving some submissions, and potential gaming of the system is revealed, which reveals that the assistant could be manipulated to enhance scores through fabricated justifications, highlighting potential vulnerabilities of automated review tools.

Abstract

Large language models (LLMs) represent a promising, but controversial, tool in aiding scientific peer review. This study evaluates the usefulness of LLMs in a conference setting as a tool for vetting paper submissions against submission standards. We conduct an experiment at the 2024 Neural Information Processing Systems (NeurIPS) conference, where 234 papers were voluntarily submitted to an "LLM-based Checklist Assistant." This assistant validates whether papers adhere to the author checklist used by NeurIPS, which includes questions to ensure compliance with research and manuscript preparation standards. Evaluation of the assistant by NeurIPS paper authors suggests that the LLM-based assistant was generally helpful in verifying checklist completion. In post-usage surveys, over 70% of authors found the assistant useful, and 70% indicate that they would revise their papers or checklist responses based on its feedback. While causal attribution to the assistant is not definitive, qualitative evidence suggests that the LLM contributed to improving some submissions. Survey responses and analysis of re-submissions indicate that authors made substantive revisions to their submissions in response to specific feedback from the LLM. The experiment also highlights common issues with LLMs: inaccuracy (20/52) and excessive strictness (14/52) were the most frequent issues flagged by authors. We also conduct experiments to understand potential gaming of the system, which reveal that the assistant could be manipulated to enhance scores through fabricated justifications, highlighting potential vulnerabilities of automated review tools.

Figures (13)

• Figure 1: Example of checklist questions, answers, and LLM-provided review.
• Figure 2: Summary of author checklist completion and LLM feedback.
• Figure 3: Distribution of 'Needs improvement' scores given by the Checklist Assistant, per checklist. Out of 15 questions, all participants received at least 8 'Needs improvement' and at most 13. More than half of the participants received 12 or more.
• Figure 4: Survey questions administered to the participants
• Figure 5: Responses to survey questions pre- and post-usage of the Checklist Assistant, from all authors who responded to both surveys ($n$=63). Error bars show $95\%$ confidence intervals for the sample proportion. The majority of surveyed authors reported a positive experience using the Checklist Assistant.