Mind the Blind Spots: A Focus-Level Evaluation Framework for LLM Reviews

TL;DR

The paper addresses the challenge of trusting LLM-generated reviews by introducing a focus-level evaluation framework that computes focus distributions over predefined review facets and compares them to human expert focus. It defines E(L,A,P) to produce two distributions, F^+ and F^-, for strengths and weaknesses, across a curated set of target and aspect facets, and builds an automatic annotator with high inter-annotator agreement. Using OpenReview data (ICLR 2021–2024) and a panel of 8 LLMs plus a fine-tuned model, the study reveals that LLM reviews biasedly emphasize technical validity while largely neglecting novelty, with a fine-tuned model achieving the closest alignment to human focus. The framework and dataset enable continuous, automated monitoring of LLM review behavior and offer actionable guidance for training LLMs and integrating them with human experts to improve the peer-review process across AI venues and beyond.

Abstract

Peer review underpins scientific progress, but it is increasingly strained by reviewer shortages and growing workloads. Large Language Models (LLMs) can automatically draft reviews now, but determining whether LLM-generated reviews are trustworthy requires systematic evaluation. Researchers have evaluated LLM reviews at either surface-level (e.g., BLEU and ROUGE) or content-level (e.g., specificity and factual accuracy). Yet it remains uncertain whether LLM-generated reviews attend to the same critical facets that human experts weigh -- the strengths and weaknesses that ultimately drive an accept-or-reject decision. We introduce a focus-level evaluation framework that operationalizes the focus as a normalized distribution of attention across predefined facets in paper reviews. Based on the framework, we developed an automatic focus-level evaluation pipeline based on two sets of facets: target (e.g., problem, method, and experiment) and aspect (e.g., validity, clarity, and novelty), leveraging 676 paper reviews (https://figshare.com/s/d5adf26c802527dd0f62) from OpenReview that consists of 3,657 strengths and weaknesses identified from human experts. The comparison of focus distributions between LLMs and human experts showed that the off-the-shelf LLMs consistently have a more biased focus towards examining technical validity while significantly overlooking novelty assessment when criticizing papers.

Figures (17)

• Figure 1: We introduce a focus-level evaluation framework for assessing LLM reviews, which computes focus distributions and compares them against human reviews based on predefined facets. The focus-level evaluation offers actionable insights into how to improve LLMs' paper review capability and how to most effectively leverage LLM reviews in the peer review process.
• Figure 2: The overall process of automated focus-level evaluation. We first extracted strengths and weaknesses from review data on the OpenReview platform as the expert reviews. To identify key strengths and weaknesses influencing the final acceptance, we extracted them from the meta-review and augmented details from individual reviewer comments. Each strength and weakness was then annotated with a target and aspect by our automatic annotator. Finally, we computed the focus distributions by normalizing the frequency of annotated targets and aspects, and compare this distribution with that of LLM reviews.
• Figure 3: Distribution of strengths and weaknesses. Unlike human experts, LLMs reported a consistent count regardless of paper contents. o1-mini identified the most, while Llama models identified the fewest points.
• Figure 4: A visualization of focus distributions by target/aspect and strength/weakness, in a descending order of cosine similarity. Overall, both groups showed similar view points in reviewing papers, focusing on technical targets (i.e., Method, Experiment, and Theory) and validity. However, LLMs showed a more biased focus towards the technical validity whereas human experts exhibited more balanced focus. Moreover, all the LLMs lack consideration of Novelty for weaknesses compared to human experts, which is a significant limitation in reviewing papers.
• Figure 5: Prompt for Meta-Review Summarization