Assessing the impact of Open Research Information Infrastructures using NLP driven full-text Scientometrics: A case study of the LXCat open-access platform

TL;DR

This paper addresses the problem of quantifying the influence of Open Research Information infrastructures (ORI) beyond traditional citation metrics. It proposes a full-text NLP-driven scientometric framework and applies it to the LXCat platform in low temperature plasma research, analyzing approximately 403 full-text articles citing three foundational LXCat publications. The authors extract chemical species, database mentions, BOLSIG+ solver references, topic structure with BERTopic, and country affiliations to reveal data usage patterns, solver–database coupling, and thematic evolution, supported by open-source code. The study demonstrates that LXCat functions as a mature, globally engaged epistemic infrastructure that shapes modeling practices and data workflows, with implications for infrastructure design, governance, and sustainability across research communities.

Abstract

Open research information (ORI) play a central role in shaping how scientific knowledge is produced, disseminated, validated, and reused across the research lifecycle. While the visibility of such ORI infrastructures is often assessed through citation-based metrics, in this study, we present a full-text, natural language processing (NLP) driven scientometric framework to systematically quantify the impact of ORI infrastructures beyond citation counts, using the LXCat platform for low temperature plasma (LTP) research as a representative case study. The modeling of LTPs and interpretation of LTP experiments rely heavily on accurate data, much of which is hosted on LXCat, a community-driven, open-access platform central to the LTP research ecosystem. To investigate the scholarly impact of the LXCat platform over the past decade, we analyzed a curated corpus of full-text research articles citing three foundational LXCat publications. We present a comprehensive pipeline that integrates chemical entity recognition, dataset and solver mention extraction, affiliation based geographic mapping and topic modeling to extract fine-grained patterns of data usage that reflect implicit research priorities, data practices, differential reliance on specific databases, evolving modes of data reuse and coupling within scientific workflows, and thematic evolution. Importantly, our proposed methodology is domain-agnostic and transferable to other ORI contexts, and highlights the utility of NLP in quantifying the role of scientific data infrastructures and offers a data-driven reflection on how open-access platforms like LXCat contribute to shaping research directions. This work presents a scalable scientometric framework that has the potential to support evidence based evaluation of ORI platforms and to inform infrastructure design, governance, sustainability, and policy for future development.

Figures (13)

• Figure 1: Overview of the data processing pipeline used to assess the scholarly impact of the LXCat platform. Starting from metadata extraction of articles citing three foundational LXCat publications pancheshnyi2012lxcatpitchford2017lxcatcarbone2021data, the workflow includes deduplication, full-text PDF collection, and conversion to structured formats. Cleaned textual data is used for a suite of NLP tasks, including chemical entity recognition, dataset mention detection, BOLSIG+ solver extraction, country attribution, and topic modeling. The final curated dataset enables temporal and thematic analysis of trends in LTP research.
• Figure 2: Overview of the chemical species extraction pipeline. Starting from raw text files, chemical entities are identified using ChemDataExtractor, followed by species-level filtering. Each entity is resolved to its canonical form via PubChem and aggregated to compute per-document frequencies. These are then filtered against a curated list of LXCat-relevant gases to produce standardized species dictionaries for downstream analysis.
• Figure 3: Overview of the database mention extraction pipeline. Starting from plain text files, the workflow applies sentence segmentation, followed by keyword filtering to isolate data relevant sentences. Tokens are then processed through tokenization and author name disambiguation, after which database names are identified and aggregated into document-level frequency counts.
• Figure 4: Overview of the topic modeling pipeline. Full-text documents undergo multiple preprocessing steps, including lowercasing, removal of equations, units, citations, and non-informative tokens, followed by lemmatization and frequency-based token filtering. The cleaned corpus is then passed to BERTopic. The output consists of full-text documents labeled with their most representative topic.
• Figure 5: Annual citation trends for the three foundational LXCat publications. (a) Full Scopus citation counts. (b) Citation counts for articles with full-text access used in this study for NLP based analysis.