Academic collaboration on large language model studies increases overall but varies across disciplines

TL;DR

This study empirically assesses how large language models influence interdisciplinary scientific collaboration by leveraging OpenAlex data, Shannon Entropy, and co-authorship network analyses. By comparing LLM papers to machine learning and non-LLM controls and applying a Difference-in-Difference framework around the ChatGPT release, the work shows a general rise in collaboration diversity—especially in Computer Science and Social Science—alongside field-specific patterns. Network analyses reveal CS and Medicine as central hubs with international institutions bridging disparate communities, suggesting LLMs can democratize and diversify collaboration while reinforcing existing core domains. The findings have implications for policy and institutional strategy to foster cross-disciplinary partnerships, while highlighting limitations in data quality and discipline classification that warrant further study and methodological refinement.

Abstract

Interdisciplinary collaboration is crucial for addressing complex scientific challenges. Recent advancements in large language models (LLMs) have shown significant potential in benefiting researchers across various fields. To explore their potential for interdisciplinary collaboration, we collect and analyze data from OpenAlex, an open-source academic database. Our dataset comprises 59,293 LLM-related papers, along with 70,945 machine learning (ML) papers and 73,110 papers from non-LLM/ML fields as control groups. We first employ Shannon Entropy to assess the diversity of collaboration. Our results reveal that many fields have exhibited a more significant increasing trend following the release of ChatGPT as compared to the control groups. In particular, Computer Science and Social Science display a consistent increase in both institution and department entropy. Other fields such as Decision Science, Psychology, and Health Professions have shown minor to significant increases. Our difference-in-difference analysis also indicates that the release of ChatGPT leads to a statistically significant increase in collaboration in several fields, such as Computer Science and Social Science. In addition, we analyze the author networks and find that Computer Science, Medicine, and other Computer Science-related departments are the most prominent. Regarding authors' institutions, our analysis reveals that entities such as Stanford University, Harvard University, and University College London are key players, either dominating centrality or playing crucial roles in connecting research networks. Overall, this study provides valuable information on the current landscape and evolving dynamics of collaboration networks in LLM research. It also suggests potential areas for fostering more diverse collaborations and highlights the need for continued research on the impact of LLMs on scientific practices.

Figures (8)

• Figure 1: (a) The distribution of papers in the collection by field. (b) The temporal pattern of entropy and paper count based on authors' affiliated institution (top) and department (bottom) information, respectively. In both subplots, the x-axis denotes the date by year. The primary y-axis denotes the Shannon Entropy calculated using \ref{['eq2']}, while the secondary y-axis indicates the count of papers. Each point represents the averaged entropy based on papers published in one quarter. The dotted gray line indicates when OpenAI released ChatGPT.
• Figure 2: Collaboration diversity over time based on (a) authors' institutions and (b) authors' departments. Panel (a) is based on 30,075 papers with complete institution information, while Panel (b) is based on 22,453 papers with complete department information. In both plots, the x-axis denotes the date by year, while the y-axis denotes the Shannon Entropy calculated using \ref{['eq2']}. Each point represents the monthly averaged entropy based on papers published in one quarter. The line represents the trend using a non-parametric regression technique called locally weighted scatterplot smoothing (LOWESS). Each point in the panel represents the averaged entropy based on papers published in one year. The dotted gray line implies the date when ChatGPT was released by OpenAI.
• Figure 3: The impact of ChatGPT on the entropy of LLM group as compared to (a) ML and (b) Non-LLM/ML paper groups. Coefficients and Std. Error of DiD ($\beta_3$) by field. When the confidence interval does NOT cross zero (the dashed line), the effect is considered statistically significant, implying that we can be 95% confident that ChatGPT has a real impact that isn't due to random chance.
• Figure 4: Co-authorship networks based on authors' institutional affiliations. Each node represents a researcher's institution, and each edge represents a co-authorship between pairs of researchers from the respective institutions or departments. Node colors represent the clusters to which the nodes belong, determined based on the Louvain modularity. Node labels represent the top 20% of nodes ranked by degree centrality. Edge thickness represents the frequency of co-authorship between the connected researchers.
• Figure 5: Co-authorship networks based on authors' department affiliations. Each node represents a researcher's department, and each edge represents a co-authorship between pairs of researchers from the respective institutions or departments. Node colors represent the clusters to which the nodes belong, determined based on the Louvain modularity. Node labels represent the top 20% of nodes ranked by degree centrality. Edge thickness represents the frequency of co-authorship between the connected researchers.