Citation Sentiment Reflects Multiscale Sociocultural Norms

TL;DR

The paper investigates how citation sentiment in neuroscience reflects multiscale sociocultural norms, using a large language model to classify sentiment across over 600k citation instances. It demonstrates that sentiment favors collaborators and varies with author status, disciplinary benchwork versus dry-lab approaches, and national culture factors such as power distance and individualism. The findings reveal systematic patterns linking collaboration, discipline practices, and cultural context to how scientists express approval or critique in citations, highlighting the social construction of science. This work underscores the importance of accounting for sociocultural factors when interpreting citation behavior and suggests avenues for more equitable and open scholarly communication.

Abstract

Modern science is formally structured around scholarly publication, where scientific knowledge is canonized through citation. Precisely how citations are given and accrued can provide information about the value of discovery, the history of scientific ideas, the structure of fields, and the space or scope of inquiry. Yet parsing this information has been challenging because citations are not simply present or absent; rather, they differ in purpose, function, and sentiment. In this paper, we investigate how critical and favorable sentiments are distributed across citations, and demonstrate that citation sentiment tracks sociocultural norms across scales of collaboration, discipline, and country. At the smallest scale of individuals, we find that researchers cite scholars they have collaborated with more favorably (and less critically) than scholars they have not collaborated with. Outside collaborative relationships, higher h-index scholars cite lower h-index scholars more critically. At the mesoscale of disciplines, we find that wetlab disciplines tend to be less critical than drylab disciplines, and disciplines that engage in more synthesis through publishing more review articles tend to be less critical. At the largest scale of countries, we find that greater individualism (and lesser acceptance of the unequal distribution of power) is associated with more critical sentiment. Collectively, our results demonstrate how sociocultural factors can explain variations in sentiment in scientific communication. As such, our study contributes to a broader understanding of how human factors influence the practice of science, and underscores the importance of considering the larger sociocultural contexts in which science progresses.

Figures (8)

• Figure 1: Sentiment analysis in citation and author networks. A two-layer network comprising a network of citations and a network of collaborators. In the citation network, nodes represent scientific articles, and edges represent citations. Sentiment analysis categorizes each edge as favorable (blue), neutral (grey), or critical (red). In the collaboration network, nodes represent scholars, and scholars who co-author articles are connected by edges.
• Figure 2: Citation sentiment depends on distance and years since (or to) first collaboration.(A) Citation sentiment relative to the null model as a function of the collaboration distance between the last authors of the citing and cited works. Shaded regions denote one standard deviation from the mean at a given collaboration distance. (B) Citation sentiment as a function of the number of years since the first collaboration between the two last authors. A negative year implies that at the time of a citation, the two scholars had yet to collaborate but would do so in the future. Shaded regions denote one standard deviation.
• Figure 3: Higher h-index scholars critically cite lower h-index scholars. Citation sentiment as a function of the difference in h-index between the citing and cited last authors.
• Figure 4: Critical citation sentiment varies across disciplines.(A) Mean critical sentiment in a discipline toward collaborators versus non-collaborators. (B) Mean critical sentiment as a function of the benchwork score of the citing paper's discipline. We define the benchwork score as the estimated proportion of wet lab papers (as opposed to dry lab papers) in a discipline. Orange corresponds to the sentiment toward non-collaborators, cyan to the sentiment toward collaborators, and grey to the sentiment toward both. Lines are fit using weighted least squares regression; shaded regions represent $95\%$ confidence intervals; $s$ reports the slope of the regression. (C) Mean critical sentiment as a function of the synthesis score of a citing paper's discipline. We define synthesis as the proportion of review papers in a discipline. (D) Relationship between the mean critical sentiment and bias. We define bias as the difference between sentiments toward non-collaborators and collaborators. (E) Bias in critical sentiment as a function of a discipline's benchwork score. (F) Bias in critical sentiment as a function of disciplinary synthesis.
• Figure 5: Favorable citation sentiment varies across disciplines.(A) Mean favorable sentiment in a discipline toward collaborators versus non-collaborators. (B) Mean favorable sentiment as a function of the benchwork score of the citing paper's discipline. We define benchwork score as the estimated proportion of wet lab papers (as opposed to dry lab papers) in a discipline. Orange corresponds to the sentiment toward non-collaborators, cyan to the sentiment toward collaborators, and grey to the sentiment toward both. Lines are fit using weighted least squares regression; shaded regions represent $95\%$ confidence intervals; $s$ reports the slope of the regression. (C) Mean favorable sentiment as a function of the synthesis score of a citing paper's discipline. We define synthesis as the proportion of review papers in a discipline. (D) Relationship between the mean favorable sentiment and bias. We define bias as the difference between sentiments toward non-collaborators and collaborators. (E) Bias in favorable sentiment as a function of a discipline's benchwork score. (F) Bias in favorable sentiment as a function of disciplinary synthesis.