Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Has ACL Lost Its Crown? A Decade-Long Quantitative Analysis of Scale and Impact Across Leading AI Conferences

Jianglin Ma, Ben Yao, Xiang Li, Yazhou Zhang

TL;DR

This study addresses whether ACL's dominance persists amid rapid growth in NLP/AI conferences by conducting a decade-long, cross-venue bibliometric analysis across seven leading venues. It introduces a four-dimensional framework and the Quality-Quantity Elasticity (QQE) metric to quantify how publication growth translates into scholarly impact, using data from 2014–2024 and 60,219 long papers. Key findings show machine-learning venues (ICLR, NeurIPS) sustain high, stable influence with efficient scale, NLP venues (ACL, NAACL, EMNLP) exhibit widening stratification and volatility, and AI venues (AAAI, IJCAI) display structural decline. The work provides a rigorous empirical basis for evaluating conference quality amidst expansion and offers a framework for governance and future research on research impact dynamics.

Abstract

The recent surge of language models has rapidly expanded NLP research, driving an exponential rise in submissions and acceptances at major conferences. Yet this growth has been shadowed by escalating concerns over conference quality, e.g., plagiarism, reviewer inexperience and collusive bidding. However, existing studies rely largely on qualitative accounts (e.g., expert interviews, social media discussions, etc.), lacking longitudinal empirical evidence. To fill this gap, we conduct a ten year empirical study spanning seven leading conferences. We build a four dimensional bibliometric framework covering conference scale, core citation statistics,impact dispersion, cross venue and journal influence, etc. Notably, we further propose a metric Quality Quantity Elasticity, which measures the elasticity of citation growth relative to acceptance growth. Our findings show that ML venues sustain dominant and stable impact, NLP venues undergo widening stratification with mixed expansion efficiency, and AI venues exhibit structural decline. This study provides the first decade-long, cross-venue empirical evidence on the evolution of major conferences.

Has ACL Lost Its Crown? A Decade-Long Quantitative Analysis of Scale and Impact Across Leading AI Conferences

TL;DR

This study addresses whether ACL's dominance persists amid rapid growth in NLP/AI conferences by conducting a decade-long, cross-venue bibliometric analysis across seven leading venues. It introduces a four-dimensional framework and the Quality-Quantity Elasticity (QQE) metric to quantify how publication growth translates into scholarly impact, using data from 2014–2024 and 60,219 long papers. Key findings show machine-learning venues (ICLR, NeurIPS) sustain high, stable influence with efficient scale, NLP venues (ACL, NAACL, EMNLP) exhibit widening stratification and volatility, and AI venues (AAAI, IJCAI) display structural decline. The work provides a rigorous empirical basis for evaluating conference quality amidst expansion and offers a framework for governance and future research on research impact dynamics.

Abstract

The recent surge of language models has rapidly expanded NLP research, driving an exponential rise in submissions and acceptances at major conferences. Yet this growth has been shadowed by escalating concerns over conference quality, e.g., plagiarism, reviewer inexperience and collusive bidding. However, existing studies rely largely on qualitative accounts (e.g., expert interviews, social media discussions, etc.), lacking longitudinal empirical evidence. To fill this gap, we conduct a ten year empirical study spanning seven leading conferences. We build a four dimensional bibliometric framework covering conference scale, core citation statistics,impact dispersion, cross venue and journal influence, etc. Notably, we further propose a metric Quality Quantity Elasticity, which measures the elasticity of citation growth relative to acceptance growth. Our findings show that ML venues sustain dominant and stable impact, NLP venues undergo widening stratification with mixed expansion efficiency, and AI venues exhibit structural decline. This study provides the first decade-long, cross-venue empirical evidence on the evolution of major conferences.

Paper Structure

This paper contains 16 sections, 3 equations, 9 figures, 2 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Bibliometric Studies on Conference Quality
  4. Expert- and Community-Based Investigation
  5. Methodology
  6. Data Acquisition
  7. Multi-Dimensional Bibliometric Framework
  8. Analytical Method
  9. Normalization Controls
  10. Experiment
  11. Main Results
  12. Comprehensive Empirical Decomposition of Conference Trends (RQ1–RQ7)
  13. Discussion
  14. Conclusion
  15. Limitations
  16. ...and 1 more sections

Figures (9)

  • Figure 1: Annual submissions and acceptances across major NLP/AI conferences from 2014 to 2024.
  • Figure 2: The proposed bibliometric framework.
  • Figure 3: Temporal trends of average and median citations across major Conferences (2014–2024).
  • Figure 4: The head and tail of citation impact result.
  • Figure 5: Gini coefficient trends (2014–2024).
  • ...and 4 more figures