Balancing Specialization and Adaptation in a Transforming Scientific Landscape

Abstract

How do scientists navigate between the need to capitalize on their prior knowledge through specialization, and the urge to adapt to evolving research opportunities? Drawing from diverse perspectives on adaptation, including cultural evolution, this paper proposes an unsupervised Bayesian approach motivated by Optimal Transport of the evolution of scientists' research portfolios in response to transformations in their field. The model relies on $186,162$ scientific abstracts and authorship data to evaluate the influence of intellectual, social, and institutional resources on scientists' trajectories within a cohort of $2\,094$ high-energy physicists between 2000 and 2019. Using Inverse Optimal Transport, the reallocation of research efforts is shown to be shaped by learning costs, thus enhancing the utility of the scientific capital disseminated among scientists. Two dimensions of social capital, namely ``diversity'' and ``power'', have opposite associations with the magnitude of change in scientists' research interests: while ``diversity'' is associated with greater change and expansion of research portfolios, ``power'' is associated with more stable research agendas. Social capital plays a more crucial role in shifts between cognitively distant research areas. More generally, this work suggests new approaches for understanding, measuring and modeling collective adaptation using Optimal Transport.

Figures (20)

• Figure 1: Changes in a scientist's "research portfolio" over time. Colors designate research areas. Resources entail any intellectual or methodological assets that a scientist uses to investigate problems in each area. "Conversion" repurposes knowledge to new goals. "Displacement" is the replacement of certain research interests with little or no transfer of prior knowledge, as illustrated by the green area. Layering is the introduction of new research interests via the addition of new knowledge.
• Figure 2: Initial (in blue) and late (in red) research portfolios of two physicists. Physicist A shifted their focus from neutrinos physics to dark matter. Physicist B, on the other hand, pursued a remarkably stable research agenda.
• Figure 3: \ref{['fig:model_schema_a']},\ref{['fig:model_schema_b']},\ref{['fig:model_schema_c']}: Transfers of attention across research areas.$\bm{x_a}$ and $\bm{y_a}$ are the distributions scientist $a$'s attention across research areas in two consecutive time periods. $\bm{\theta_{ak}}=(\theta_{akk'})$ represents the fraction of the attention devoted by an author $a$ to a topic $k$ redirected to topics $k' \in \{1,\dots,K\}$, as scientists repurpose, expand or concentrate their knowledge. By definition, $\sum_{k'} \theta_{akk'}=1$. Figure \ref{['fig:model_structure']}: Hierarchical model. $\theta_a$ is drawn from a hierarchical process, with intellectual and social capital ($\bm{I_a}$ and $\bm{S_a}$) as covariates. Observed variables are represented in gray, latent variables in white.
• Figure 4: (a) Aggregate transfers of attention across research areas, between 2000-2009 (to the left) and 2015-2019 (to the right). Widths of flows are proportional to $\sum_a X_{ak}\theta_{akk'}$. Insignificant transfers (that happen less than expected by chance alone assuming uniform mixing) are transparent. (b) For purposes of clarity, the same figure is repeated to the right, highlighting only the flows directed towards "Dark matter".
• Figure 5: Aggregate transfers of attention throughout the years 2000 to 2019, considering four time-intervals of five years each. $\mu(D)$ is the average portfolio diversity of the cohort during each time-period.