Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Towards a valid bibliometric measure of epistemic breadth of researchers

Paul Donner, Clemens Blümel

TL;DR

This paper addresses the lack of validated metrics for epistemic breadth by proposing a knowledge-space framework that positions each publication in a semantic space via SPECTER embeddings, enabling distances that reflect thematic relatedness. It defines and tests multiple breadth indicators, with the weighted furthest-neighbor measure showing strong external validation against discipline-switching researchers and coherent internal validation via self-citation patterns. The study demonstrates that simple average pairwise similarities already discriminate epistemic breadth, and that robustness-enhanced measures offer practical advantages for handling topical outliers. While promising, the authors acknowledge limitations in data scope and embedding methods, and call for broader validation across disciplines and time. Overall, this work provides the first empirically validated bibliometric indicator of epistemic breadth and establishes a flexible, future-proof framework for measuring researchers’ cognitive portfolios.

Abstract

The concept of epistemic breadth of the work of a researcher refers to the scope of their knowledge claims, as reflected in published research reports. Studies of epistemic breadth have been hampered by the lack of a validated measure of the concept. Here we introduce a knowledge space approach to the measurement of epistemic breadth and propose to use the semantic similarity network of an author's publication record to operationalize a measure. In this approach, each paper has its own location in a common abstract vector space based on its content. Proximity in knowledge space corresponds to thematic similarity of publications. Candidate measures of epistemic breadth derived from aggregate similarity values of researchers' bodies of work are tested against external validation data of researchers known to have made a major change in research topic and against self-citation data. We find that some candidate measures co-vary well with known epistemic breadth of researchers in the empirical data and can serve as valid indicators of the concept.

Towards a valid bibliometric measure of epistemic breadth of researchers

TL;DR

This paper addresses the lack of validated metrics for epistemic breadth by proposing a knowledge-space framework that positions each publication in a semantic space via SPECTER embeddings, enabling distances that reflect thematic relatedness. It defines and tests multiple breadth indicators, with the weighted furthest-neighbor measure showing strong external validation against discipline-switching researchers and coherent internal validation via self-citation patterns. The study demonstrates that simple average pairwise similarities already discriminate epistemic breadth, and that robustness-enhanced measures offer practical advantages for handling topical outliers. While promising, the authors acknowledge limitations in data scope and embedding methods, and call for broader validation across disciplines and time. Overall, this work provides the first empirically validated bibliometric indicator of epistemic breadth and establishes a flexible, future-proof framework for measuring researchers’ cognitive portfolios.

Abstract

The concept of epistemic breadth of the work of a researcher refers to the scope of their knowledge claims, as reflected in published research reports. Studies of epistemic breadth have been hampered by the lack of a validated measure of the concept. Here we introduce a knowledge space approach to the measurement of epistemic breadth and propose to use the semantic similarity network of an author's publication record to operationalize a measure. In this approach, each paper has its own location in a common abstract vector space based on its content. Proximity in knowledge space corresponds to thematic similarity of publications. Candidate measures of epistemic breadth derived from aggregate similarity values of researchers' bodies of work are tested against external validation data of researchers known to have made a major change in research topic and against self-citation data. We find that some candidate measures co-vary well with known epistemic breadth of researchers in the empirical data and can serve as valid indicators of the concept.

Paper Structure

This paper contains 19 sections, 3 figures, 1 table.

Table of Contents

  1. Introduction
  2. Literature review
  3. Work using classication systems
  4. Work using publication clustering
  5. Critical assessment
  6. Methods and data
  7. Approach
  8. Publication data
  9. SPECTER semantic text embeddings
  10. Validation of candidate measures with external data
  11. Validation of measure with internal data
  12. Candidate measures
  13. Results
  14. Qualitative assessment of the soundness of the knowledge space approach by visualizations of key knowledge space sections
  15. Average similarities of discipline-switching and control authors' papers
  16. ...and 4 more sections

Figures (3)

  • Figure 1: Example MDS visualization of six matched treatment-control author pairs' paper positions in local knowledge space. Labels: T: treatment researcher publications (changed discipline). C: Matched control researcher publications.
  • Figure 2: MDS visualization of all treatment-control author pairs' paper positions in local knowledge space. Labels: T: treatment researcher publications (changed discipline). C: Matched control researcher publications.
  • Figure 3: Average cosine similarity of publications of CDF and matched control authors. Lines indicate matched pairs.