Do Researchers Benefit Career-wise from Involvement in International Policy Guideline Development?

Abstract

Researchers are no longer limited to producing knowledge; in today's complex world, they also address societal challenges by engaging in policymaking. Although involvement in policymaking has expanded, direct empirical evidence of its career benefits remains underexplored. Prior survey-based studies suggest potential advantages-such as broader professional networks and enhanced opportunities-yet raise concerns about insufficient institutional support. Here, we examine the 2021 WHO global air quality guideline-a science-based regulatory guideline-as a case study. To evaluate the impact of guideline development on research outcomes, we match guideline researchers with a control group of peers sharing similar research topics and prior performance. Our analysis reveals that guideline researchers attain higher future citation counts in both academic and policy domains. New collaborations formed during development yield publications with higher citation impact and the disruptive index. Moreover, about half the guideline's references are derived from guideline researchers' papers, highlighting their central role in shaping the evidence base. These results provide empirical support for the career benefits of policy engagement. Our findings indicate that engaging in international guideline development offers tangible career incentives for researchers, and that institutions can enhance research impact and promote innovative scientific progress by actively supporting their researchers' participation in such initiatives.

Figures (7)

• Figure 1: a Flow diagram of the matching procedure used to match guideline researchers with control researchers. In Step 1, each guideline researcher's specialization vector is computed by averaging 1536-dimensional text embeddings derived from titles or abstracts of the papers authored by them in Scopus (2011--2015). In Step 2, the top 20 non-guideline researchers with the highest cosine similarity to each specialization vector are retrieved. In Step 3, the mean squared error (MSE) is calculated based on academic citations (2011--2015), IGO policy citations (as of 2015), and publication counts (as of 2015), and the candidate with the lowest MSE is selected as the matched control researcher. b--c Trends in cumulative academic and IGO policy citations for the treatment and control groups from 2011 to 2021. Orange marks the guideline development period (2016--2021). The plots show the median values for each group per year, with error bars indicating interquartile range. ** $p<0.01$, * $p<0.05$.
• Figure 2: Co-authorship networks and attribute assortativity of guideline researchers. Node colors indicate roles: blue for the Guideline Development Group, green for the Systematic Review Team, brown for External Methodologists, and red for the External Review Group. a The co-authorship network constructed from publications spanning 1970--2015 (n=102). b The co-authorship network during the guideline development period (2016--2021) (n=104). c Total inter-attribute edge counts in the 1970--2015 network. d Total inter-attribute edge counts in the 2016--2021 network. e Comparison of attribute assortativity between the two periods networks, showing an increase from 0.028 (1970--2015) to 0.101 (2016--2021) with a borderline significant difference (p = 0.069).
• Figure 3: Trends in collaboration types among guideline researchers, treatment, and control groups. a--b show the total number of papers published each year (stacked by collaboration-initiation period), while c--d include only those papers co-authored within each respective group. a, c represent the treatment group ($n=101$), and b, d the control group ($n=101$). Vertical dashed lines mark the official guideline development window (2016--2021). Although the overall publication output increased for all groups, the treatment group (c) show a marked surge in new intra-group collaborations (orange) after 2016, in contrast to the control group (d).
• Figure 4: Impact of collaboration types on citation and disruption for papers published between 2016 and 2021 (the guideline development period). a Log-transformed two-year citation counts $\log_{10}(C_{2}+1)$ for papers in each collaboration type (Non_collab, Pre_guideline_collab, During_guideline_collab). b Disruptive index $(D)$ for the same collaboration types. Each violin plot shows the distribution for all papers published between 2016 and 2021 in that group, with individual points representing single publications. *** $p<0.001$, ** $p<0.01$, * $p<0.0167$.
• Figure S1: Geographic distribution and citation counts of the guideline reference papers. a Institutional affiliation by country for guideline researchers, colored by geographic region (n=101). b Publication year distribution of academic papers cited in the guideline (n=183), with guideline development period (2016-2021) highlighted in orange.