Consistency Analysis of ChatGPT
Myeongjun Erik Jang, Thomas Lukasiewicz
TL;DR
The paper systematically evaluates the logical consistency of ChatGPT and GPT-4 across semantic, negation, symmetric, and transitive dimensions using the BECEL benchmark. It shows that despite advances, both models frequently violate basic consistency properties and that prompt design, few-shot learning, or simply scaling do not reliably resolve these issues. The authors also explore explainability, revealing that generated explanations can be self-contradictory or inaccurate. The study highlights significant trust and safety concerns for real-world deployment and suggests hybrid approaches and more rigorous evaluation of reasoning and explanations as future directions.
Abstract
ChatGPT has gained a huge popularity since its introduction. Its positive aspects have been reported through many media platforms, and some analyses even showed that ChatGPT achieved a decent grade in professional exams, adding extra support to the claim that AI can now assist and even replace humans in industrial fields. Others, however, doubt its reliability and trustworthiness. This paper investigates the trustworthiness of ChatGPT and GPT-4 regarding logically consistent behaviour, focusing specifically on semantic consistency and the properties of negation, symmetric, and transitive consistency. Our findings suggest that while both models appear to show an enhanced language understanding and reasoning ability, they still frequently fall short of generating logically consistent predictions. We also ascertain via experiments that prompt designing, few-shot learning and employing larger large language models (LLMs) are unlikely to be the ultimate solution to resolve the inconsistency issue of LLMs.