Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Benchmark of Benchmarks: Unpacking Influence and Code Repository Quality in LLM Safety Benchmarks

Junjie Chu, Xinyue Shen, Ye Leng, Michael Backes, Yun Shen, Yang Zhang

TL;DR

It is found that benchmark papers show no significant advantage in academic influence over non-benchmark papers, and a key misalignment is uncovered: while author prominence correlates with paper influence, neither author prominence nor paper influence shows a significant correlation with code quality.

Abstract

The rapid growth of research in LLM safety makes it hard to track all advances. Benchmarks are therefore crucial for capturing key trends and enabling systematic comparisons. Yet, it remains unclear why certain benchmarks gain prominence, and no systematic assessment has been conducted on their academic influence or code quality. This paper fills this gap by presenting the first multi-dimensional evaluation of the influence (based on five metrics) and code quality (based on both automated and human assessment) on LLM safety benchmarks, analyzing 31 benchmarks and 382 non-benchmarks across prompt injection, jailbreak, and hallucination. We find that benchmark papers show no significant advantage in academic influence (e.g., citation count and density) over non-benchmark papers. We uncover a key misalignment: while author prominence correlates with paper influence, neither author prominence nor paper influence shows a significant correlation with code quality. Our results also indicate substantial room for improvement in code and supplementary materials: only 39% of repositories are ready-to-use, 16% include flawless installation guides, and a mere 6% address ethical considerations. Given that the work of prominent researchers tends to attract greater attention, they need to lead the effort in setting higher standards.

Benchmark of Benchmarks: Unpacking Influence and Code Repository Quality in LLM Safety Benchmarks

TL;DR

It is found that benchmark papers show no significant advantage in academic influence over non-benchmark papers, and a key misalignment is uncovered: while author prominence correlates with paper influence, neither author prominence nor paper influence shows a significant correlation with code quality.

Abstract

The rapid growth of research in LLM safety makes it hard to track all advances. Benchmarks are therefore crucial for capturing key trends and enabling systematic comparisons. Yet, it remains unclear why certain benchmarks gain prominence, and no systematic assessment has been conducted on their academic influence or code quality. This paper fills this gap by presenting the first multi-dimensional evaluation of the influence (based on five metrics) and code quality (based on both automated and human assessment) on LLM safety benchmarks, analyzing 31 benchmarks and 382 non-benchmarks across prompt injection, jailbreak, and hallucination. We find that benchmark papers show no significant advantage in academic influence (e.g., citation count and density) over non-benchmark papers. We uncover a key misalignment: while author prominence correlates with paper influence, neither author prominence nor paper influence shows a significant correlation with code quality. Our results also indicate substantial room for improvement in code and supplementary materials: only 39% of repositories are ready-to-use, 16% include flawless installation guides, and a mere 6% address ethical considerations. Given that the work of prominent researchers tends to attract greater attention, they need to lead the effort in setting higher standards.
Paper Structure (46 sections, 1 equation, 19 figures, 8 tables)

This paper contains 46 sections, 1 equation, 19 figures, 8 tables.

Table of Contents

  1. Introduction
  2. Data Under Study
  3. Scope of This Study
  4. Data Collection
  5. Potential Factors
  6. Influence of Benchmark Papers (RQ1)
  7. Methodology of Influence Evaluation
  8. Influence Evaluation Results of Benchmark Papers
  9. Factors in Relation to the Influence of Benchmark Papers
  10. Quality of Benchmark Papers' Code Repositories (RQ2)
  11. Methodology of Tool-Based Evaluation
  12. Methodology of Human-Based Evaluation
  13. Tool-Based Evaluation Results of Benchmark Papers
  14. Human-Based Evaluation Results of Benchmark Papers
  15. Factors in Relation to the Code Quality of Benchmark Papers
  16. ...and 31 more sections

Figures (19)

  • Figure 1: Data collection pipeline.
  • Figure 2: Human-based evaluation results of code quality.
  • Figure 3: Human-based evaluation results of supplementary materials. Repositories without code or unavailable ones are labeled "Not Applicable." For install guides: repositories are categorized as "No" (no guide), "Partial" (guide with setup issues), or "Yes" (fully functional guide); if a guide includes all required Python/library/package versions, it is labeled "Yes: no flaws"; if version information is missing but the guide is still runnable, it is "Yes: lack versions."
  • Figure 4: PRISMA-style flow diagram for benchmark selection.
  • Figure 5: Average values of five influence-related metrics on benchmark and non-benchmark papers.
  • ...and 14 more figures