RealCritic: Towards Effectiveness-Driven Evaluation of Language Model Critiques

TL;DR

RealCritic introduces a closed-loop benchmark to evaluate language model critiques by linking critique quality to the effectiveness of subsequent corrections. It systematically studies self-critique, cross-critique, and iterative critique across eight challenging reasoning tasks, using a mix of open-source and proprietary models. Across experiments, reasoning-based models like o1-mini show superior critique performance, with cross-critique delivering larger gains on basic tasks but mixed results on specialized domains, and iterative critique revealing model-dependent dynamics. The benchmark and findings provide a resource to guide future development of critique capabilities and the code is available at the given GitHub.

Abstract

Critiques are important for enhancing the performance of Large Language Models (LLMs), enabling both self-improvement and constructive feedback for others by identifying flaws and suggesting improvements. However, evaluating the critique capabilities of LLMs presents a significant challenge due to the open-ended nature of the task. In this work, we introduce a new benchmark designed to assess the critique capabilities of LLMs. Unlike existing benchmarks, which typically function in an open-loop fashion, our approach employs a closed-loop methodology that evaluates the quality of corrections generated from critiques. Moreover, the benchmark incorporates features such as self-critique, cross-critique, and iterative critique, which are crucial for distinguishing the abilities of advanced reasoning models from more classical ones. We implement this benchmark using eight challenging reasoning tasks. We have several interesting findings. First, despite demonstrating comparable performance in direct chain-of-thought generation, classical LLMs significantly lag behind the advanced reasoning-based model o1-mini across all critique scenarios. Second, in self-critique and iterative critique settings, classical LLMs may even underperform relative to their baseline capabilities. We hope that this benchmark will serve as a valuable resource to guide future advancements. The code and data are available at \url{https://github.com/tangzhy/RealCritic}.

Figures (9)

• Figure 1: Benchmark results for self-critique and cross-critique abilities of representative LLMs. Our findings reveal a clear seperation between classical LLMs and reasoning-based LLMs: o1-mini is the only model demonstrating improvement in self-critique tasks, while also achieving the most significant gains in cross-critique performance.
• Figure 2: Examples illustrating the limitations of CriticBench lin2024criticbench: it incorrectly classifies a low-quality critique as high-quality by relying on the accuracy of predicting the input solution as the metric for critique quality. In contrast, our RealCritic accurately identifies high-quality critiques based on their effectiveness in guiding the generation of improved solutions.
• Figure 3: Comparison between the evaluation method used in CriticBench and our framework, RealCritic. Our framework operates in a closed-loop manner by assessing the quality of a critic through the quality of the new solution generated based on its feedback.
• Figure 4: Data collection process for constructing solutions to assess the critique abilities of LLMs.
• Figure 5: Performance of C→I and I→C in self-critique and cross-critique scenarios. Here, "C" denotes correct, "I" denotes incorrect, and the arrow indicates the accuracy change from the input solution to the correction after critique.