Beyond Citations: Measuring Idea-level Knowledge Diffusion from Research to Journalism and Policy-making

TL;DR

This paper develops a novel framework to measure idea-level knowledge diffusion from research to journalism and policy-making, focusing on 33 named media-effects concepts. Using a large, cross-domain corpus (2000–2019) and an embedding-regression methodology, it assesses how concept usage and meaning shift across domains and over time, beyond direct citations. The findings reveal heterogeneous diffusion, with policy usage typically diverging more from research than journalism, and demonstrate that ideas transition from theoretical constructs in research to interpretive tools in news and more applied, administrative roles in policy. The study offers a generalizable approach to quantify diffusion in terms of contextual meaning and supports broader assessments of social science impact beyond traditional citation metrics.

Abstract

Despite the importance of social science knowledge for various stakeholders, measuring its diffusion into different domains remains a challenge. This study uses a novel text-based approach to measure the idea-level diffusion of social science knowledge from the research domain to the journalism and policy-making domains. By doing so, we expand the detection of knowledge diffusion beyond the measurements of direct references. Our study focuses on media effects theories as key research ideas in the field of communication science. Using 72,703 documents (2000-2019) from three domains (i.e., research, journalism, and policy-making) that mention these ideas, we count the mentions of these ideas in each domain, estimate their domain-specific contexts, and track and compare differences across domains and over time. Overall, we find that diffusion patterns and dynamics vary considerably between ideas, with some ideas diffusing between other domains, while others do not. Based on the embedding regression approach, we compare contextualized meanings across domains and find that the distances between research and policy are typically larger than between research and journalism. We also find that ideas largely shift roles across domains - from being the theories themselves in research to sense-making in news to applied, administrative use in policy. Over time, we observe semantic convergence mainly for ideas that are practically oriented. Our results characterize the cross-domain diffusion patterns and dynamics of social science knowledge at the idea level, and we discuss the implications for measuring knowledge diffusion beyond citations.

Figures (11)

• Figure 1: Prevalence of the media effects theories across three domains. The graph shows the number of documents mentioning the relevant concepts from 2000 to 2019. Each circle in the graph denotes at least one document in a given year and a specific domain that mentions the concepts. The size of the circle is proportional to the number of documents.
• Figure 2: Corpus size per concept across domains and time. Here, the x-axis shows the publication year, and the y-axis shows the number of retrieved documents.
• Figure 3: Embedding-based distance across domains and time Here, the x-axis shows the publication year, and the y-axis shows the normed $\beta$.
• Figure 4: Embedding-based distance across domains and time Here, the x-axis shows the publication year, and the y-axis shows the normed $\beta$.
• Figure 5: Embedding-based distance across domains and time Here, the x-axis shows the publication year, and the y-axis shows the normed $\beta$.