Keeping Score: A Quantitative Analysis of How the CHI Community Appreciates Its Milestones

TL;DR

This study investigates how the CHI community engages with its intellectual heritage from 1981 to 2024 by tracing citations to milestones, modeling forgetting curves, and introducing the Milestone Coefficient to quantify long-term impact beyond raw counts. It identifies a three-part milestone typology, documents a clear decline in attention to older milestones, and reveals a Matthew effect in milestone authors. A case study on M1’s Thematic Analysis exemplifies a true super milestone whose influence persists despite evolving practice. The findings offer practical insights for sustaining meaningful, method-driven contributions in HCI and for broader meta-research in scholarly memory and recognition.

Abstract

The ACM CHI Conference has a tradition of citing its intellectual heritage. At the same time, we know CHI is highly diverse and evolving. In this highly dynamic context, it is not clear how the CHI community continues to appreciate its milestones (within and outside of CHI). We present an investigation into how the community's citations to milestones have evolved over 43 years of CHI Proceedings (1981-2024). Forgetting curves plotted for each year suggest that milestones are slowly fading from the CHI community's collective memory. However, the picture is more nuanced when we trace citations to the top-cited milestones over time. We identify three distinct types of milestones cited at CHI, a typology of milestone contributions, and define the Milestone Coefficient as a metric to assess the impact of milestone papers on a continuous scale. Further, we provide empirical evidence of a Matthew effect at CHI. We discuss the broader ramifications for the CHI community and the field of HCI.

Figures (7)

• Figure 1: Relative number of citations to CHI's top-100 most cited milestones over time. A clustering analysis reveals three types of milestones in CHI: one "super milestone" continues to grow in popularity (red), some past super milestones are fading from the CHI community's memory (light blue), and some contemporary milestones are receiving continued attention, although at lower levels (blue).
• Figure 2: CHI's forgetting curves have become "smoother" over time, lacking the visible peaks that are characteristic of milestones.
• Figure 3: Cumulative citations to the prior 300 most cited milestones from selected CHI proceedings years.
• Figure 4: Absolute (top) and relative (bottom) number of citations from selected CHI proceedings to the 300 most cited milestones published in years prior to the given proceedings year. Note that for 2002 and 2004, the number of milestones is marginally lower. Standard deviation, kurtosis, and skewness values are listed in each plot. Looking at the panels, we can identify that while the absolute number of citations to prior milestones has increased, the relative number of citations to prior milestones has decreased significantly. In other terms, milestones have lost influence over time at CHI, relative to all other citations in a given proceedings year.
• Figure 5: Milestone coefficient (MC) for the 3,000 papers most cited by CHI authors. The year corresponds to the milestone's publication year, and the size of the bubble reflects the total number of citations to the milestone from CHI papers (over all CHI proceedings). Papers below the horizontal line tend to being forgotten by the CHI community over time, while papers above the line continue to grow in citations. The distance from the dashed horizontal line indicates the milestone's impact, with high-performing milestones having a greater positive distance from the line. The plot also demonstrates the exceptional performance of the super milestone published in 2006.