Automatically detecting scientific political science texts from a large general document index
Nina Smirnova
TL;DR
The study tackles filtering large, heterogeneous BASE metadata to identify political science texts by combining a hard keyword-based filter with soft BERT-based classifiers. It introduces an English SSciBERT_politics model and a multilingual bert-base-ml-politics model, both trained with semi-automated labeling on BASE and POLLUX abstracts. Evaluations show the multilingual BERT approach achieving near 0.98 accuracy, outperforming the English model and the keyword filter, which itself reaches about 0.87 accuracy when using full metadata; the keyword-only variant is faster but less accurate. The work demonstrates effective domain-filtering for large bibliographic indexes and suggests generalization to other disciplines, enabling more efficient domain-specific literature curation and discovery.
Abstract
This technical report outlines the filtering approach applied to the collection of the Bielefeld Academic Search Engine (BASE) data to extract articles from the political science domain. We combined hard and soft filters to address entries with different available metadata, e.g. title, abstract or keywords. The hard filter is a weighted keyword-based approach. The soft filter uses a multilingual BERT-based classification model, trained to detect scientific articles from the political science domain. We evaluated both approaches using an annotated dataset, consisting of scientific articles from different scientific domains. The weighted keyword-based approach achieved the highest total accuracy of 0.88. The multilingual BERT-based classification model was fine-tuned using a dataset of 14,178 abstracts from scientific articles and reached the highest total accuracy of 0.98. The proposed filtering approach can be applied for filtering metadata from other scientific domains and therefore improve the overview of the domain-related literature and facilitate efficiency in research.