Past, Present, and Future of Citation Practices in HCI

TL;DR

The paper analyzes how a 2016 CHI policy shift lifting page limits on references coincided with a sustained, near-linear rise in mean references per CHI article, forecasting around $129.5$ references by 2030. Using data from 11,542 CHI articles (1981–2024) and an event-study design, it isolates the policy as a key driver while examining co-occurring factors such as arXiv citations, data/code repository mentions, literature reviews, collaboration, and predatory publishers. The study finds a significant post-policy change shift in citation practices, with predatory-publisher citations contributing notably, though several factors like arXiv and repositories show nuanced effects; awards and collaboration alone do not fully explain the trend. The authors discuss implications for authors, reviewers, and policy-makers, and propose potential remedies including policy caps, AI-assisted literature tools, revised peer review processes, and pre-registered reference limits to foster more sustainable scholarly communication in HCI.

Abstract

Science is a complex system comprised of many scientists who individually make decisions that, due to the size and nature of the academic system, largely do not affect the system as a whole. However, certain decisions at the meso-level of research communities, such as the Human-Computer Interaction (HCI) community, may result in deep and long-lasting behavioral changes in scientists. In this article, we provide empirical evidence on how a change in editorial policies introduced at the ACM CHI Conference in 2016 destabilized the CHI research community and launched it on an expansive path, denoted by a year-by-year increase in the mean number of references included in CHI articles. If this near-linear trend continues undisrupted, an article at CHI 2030 will include on average almost 130 references. The trend toward more citations reflects a citation culture where quantity is prioritized over quality, contributing to both author and peer reviewer fatigue. Our exploratory analysis highlights the profound impact of meso-level policy adjustments on the evolution of scientific fields and disciplines, urging all stakeholders to carefully consider the broader implications of such changes.

Figures (13)

• Figure 1: Number of references per article in the CHI Conference Proceedings from 1981 to 2024.
• Figure 2: Excerpts from the Call for Papers (CfPs) at the CHI Conference in 2015 and 2016 (own highlighting).
• Figure 3: Extrapolation of the observed trend in the mean (red) and minimum (blue) number of references per CHI article. CHI articles are predicted to include on average almost 130 references in the year 2030. The minimum number of references per CHI article is estimated to reach 33.7 references per article in 2030, which is more than the 2015 average level of references.
• Figure 4: The number of co-authors in CHI papers has increased over time, highlighting the trend toward increased collaboration in HCI.
• Figure 5: Mean number of references per author in CHI articles