Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

On Tables with Numbers, with Numbers

Konstantinos Kogkalidis, Stergios Chatzikyriakidis

TL;DR

This paper argues against tables with numbers on the basis of their epistemic irrelevance, their environmental impact, their role in enabling and exacerbating social inequalities, and their deep ties to commercial applications and profit-driven research.

Abstract

This paper is a critical reflection on the epistemic culture of contemporary computational linguistics, framed in the context of its growing obsession with tables with numbers. We argue against tables with numbers on the basis of their epistemic irrelevance, their environmental impact, their role in enabling and exacerbating social inequalities, and their deep ties to commercial applications and profit-driven research. We substantiate our arguments with empirical evidence drawn from a meta-analysis of computational linguistics research over the last decade.

On Tables with Numbers, with Numbers

TL;DR

This paper argues against tables with numbers on the basis of their epistemic irrelevance, their environmental impact, their role in enabling and exacerbating social inequalities, and their deep ties to commercial applications and profit-driven research.

Abstract

This paper is a critical reflection on the epistemic culture of contemporary computational linguistics, framed in the context of its growing obsession with tables with numbers. We argue against tables with numbers on the basis of their epistemic irrelevance, their environmental impact, their role in enabling and exacerbating social inequalities, and their deep ties to commercial applications and profit-driven research. We substantiate our arguments with empirical evidence drawn from a meta-analysis of computational linguistics research over the last decade.
Paper Structure (18 sections, 3 figures)

This paper contains 18 sections, 3 figures.

Table of Contents

  1. Introduction
  2. The Multiple Facets of Number
  3. Number as Virtue
  4. Number as Number
  5. Number as Nothing
  6. Number as Vice
  7. Resource Exhaustion
  8. No NLP to Be Done on a Dead Planet
  9. Institutional Bias & Privilege
  10. Science of the Few
  11. Science for the Few
  12. From Inequality to Alienation
  13. Science & Profit
  14. Stand on the Shoulders of (Tech) Giants
  15. Research (and Development)
  16. ...and 3 more sections

Figures (3)

  • Figure 1: Box- and swarm-plots of the distribution of the number of experimental results per paper, grouped by year. We manually count the number of numbers within tables from the 50 most cited papers per year. We do not include numbers that pertain to descriptive dataset statistics, nor numbers reporting dispersion statistics (e.g., confidence intervals, standard deviations etc.). The pattern indicates a marked upwards trend over time. Most (75%) contemporary papers contain 100 to 300 numbers, while some (25%) contain up to 1 000.
  • Figure 2: Contemporary model training costs compared to the total annual R&D budgets of select U.S. institutions. The cost of training a large model is comparable to the budget of a university in the top 15th percentile, which is two orders of magnitude larger than the median budget. Budget data sourced from the 2022 report by the US National Center for Science and Engineering Statisticsa. Model cost estimates from epoch2023aitrends.
  • Figure 3: Major sponsors of the main ACL conferences over the last 10 years. To convert tiered participation counts to contributions, we assign a weight of 1 to the year's top tier, and divide the weight of each consecutive sponsorship tier by 2. The treasurer of the ACL did not respond to our request for accurate donation figures.