Automated Annotation of Scientific Texts for ML-based Keyphrase Extraction and Validation

TL;DR

Results show that the proposed label assignment approaches can generate both generic and highly specific text labels for the unlabeled texts, with up to 44% of the labels matching with those suggested by a ML keyword extraction algorithm.

Abstract

Advanced omics technologies and facilities generate a wealth of valuable data daily; however, the data often lacks the essential metadata required for researchers to find and search them effectively. The lack of metadata poses a significant challenge in the utilization of these datasets. Machine learning-based metadata extraction techniques have emerged as a potentially viable approach to automatically annotating scientific datasets with the metadata necessary for enabling effective search. Text labeling, usually performed manually, plays a crucial role in validating machine-extracted metadata. However, manual labeling is time-consuming; thus, there is an need to develop automated text labeling techniques in order to accelerate the process of scientific innovation. This need is particularly urgent in fields such as environmental genomics and microbiome science, which have historically received less attention in terms of metadata curation and creation of gold-standard text mining datasets. In this paper, we present two novel automated text labeling approaches for the validation of ML-generated metadata for unlabeled texts, with specific applications in environmental genomics. Our techniques show the potential of two new ways to leverage existing information about the unlabeled texts and the scientific domain. The first technique exploits relationships between different types of data sources related to the same research study, such as publications and proposals. The second technique takes advantage of domain-specific controlled vocabularies or ontologies. In this paper, we detail applying these approaches for ML-generated metadata validation. Our results show that the proposed label assignment approaches can generate both generic and highly-specific text labels for the unlabeled texts, with up to 44% of the labels matching with those suggested by a ML keyword extraction algorithm.

Figures (6)

• Figure 1: Potential approaches for validating ML-generated keywords for unlabeled texts when human labels are unavailable. Our proposed approaches are shown in green.
• Figure 2: SciKey's metadata generation pipeline and sumodules.
• Figure 3: Overview of the automated labeling and metadata generation process. Built on top of the SciKey module, the automated label generation process (shown in blue) takes in a text blob containing semantic information and a set of labels that are used for ML-generated keyword validation. Based on these labels, Scikey outputs a set of validated ML keywords (i.e "good" labels) and quantitative metrics reflecting keyword quality.
• Figure 4: Flowchart of artifact linkage process for label generation
• Figure 5: Flowchart of ontology-based process for label generation