Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Integrated ensemble of BERT- and features-based models for authorship attribution in Japanese literary works

Taisei Kanda, Mingzhe Jin, Wataru Zaitsu

TL;DR

This study addresses authorship attribution in Japanese literary works under small-sample constraints by introducing an integrated ensemble that fuses पांच BERT models with traditional stylometric features and classifier outputs using soft voting. Through experiments on two corpora of 10 authors each, the authors demonstrate that BERT is effective for short texts, with pre-training data substantially influencing performance. The integrated ensemble—comprising five BERTs, three feature types, and two classifiers—achieves the highest F1 scores and outperforms single models by substantial margins, including roughly a 14-point gain on a corpus not present in pre-training. The findings provide a practical framework for leveraging PLMs alongside traditional features in low-resource authorship attribution tasks, with implications for broader text classification settings.

Abstract

Traditionally, authorship attribution (AA) tasks relied on statistical data analysis and classification based on stylistic features extracted from texts. In recent years, pre-trained language models (PLMs) have attracted significant attention in text classification tasks. However, although they demonstrate excellent performance on large-scale short-text datasets, their effectiveness remains under-explored for small samples, particularly in AA tasks. Additionally, a key challenge is how to effectively leverage PLMs in conjunction with traditional feature-based methods to advance AA research. In this study, we aimed to significantly improve performance using an integrated integrative ensemble of traditional feature-based and modern PLM-based methods on an AA task in a small sample. For the experiment, we used two corpora of literary works to classify 10 authors each. The results indicate that BERT is effective, even for small-sample AA tasks. Both BERT-based and classifier ensembles outperformed their respective stand-alone models, and the integrated ensemble approach further improved the scores significantly. For the corpus that was not included in the pre-training data, the integrated ensemble improved the F1 score by approximately 14 points, compared to the best-performing single model. Our methodology provides a viable solution for the efficient use of the ever-expanding array of data processing tools in the foreseeable future.

Integrated ensemble of BERT- and features-based models for authorship attribution in Japanese literary works

TL;DR

This study addresses authorship attribution in Japanese literary works under small-sample constraints by introducing an integrated ensemble that fuses पांच BERT models with traditional stylometric features and classifier outputs using soft voting. Through experiments on two corpora of 10 authors each, the authors demonstrate that BERT is effective for short texts, with pre-training data substantially influencing performance. The integrated ensemble—comprising five BERTs, three feature types, and two classifiers—achieves the highest F1 scores and outperforms single models by substantial margins, including roughly a 14-point gain on a corpus not present in pre-training. The findings provide a practical framework for leveraging PLMs alongside traditional features in low-resource authorship attribution tasks, with implications for broader text classification settings.

Abstract

Traditionally, authorship attribution (AA) tasks relied on statistical data analysis and classification based on stylistic features extracted from texts. In recent years, pre-trained language models (PLMs) have attracted significant attention in text classification tasks. However, although they demonstrate excellent performance on large-scale short-text datasets, their effectiveness remains under-explored for small samples, particularly in AA tasks. Additionally, a key challenge is how to effectively leverage PLMs in conjunction with traditional feature-based methods to advance AA research. In this study, we aimed to significantly improve performance using an integrated integrative ensemble of traditional feature-based and modern PLM-based methods on an AA task in a small sample. For the experiment, we used two corpora of literary works to classify 10 authors each. The results indicate that BERT is effective, even for small-sample AA tasks. Both BERT-based and classifier ensembles outperformed their respective stand-alone models, and the integrated ensemble approach further improved the scores significantly. For the corpus that was not included in the pre-training data, the integrated ensemble improved the F1 score by approximately 14 points, compared to the best-performing single model. Our methodology provides a viable solution for the efficient use of the ever-expanding array of data processing tools in the foreseeable future.

Paper Structure

This paper contains 19 sections, 5 equations, 2 figures, 5 tables.

Table of Contents

  1. Introduction
  2. Related research
  3. Comparison of PLMs and classifiers
  4. Effect of pre-training data on the BERT model
  5. Ensemble of BERTs
  6. Ensemble of BERT and features
  7. Experiment
  8. Features and classifiers used
  9. Selected PLMs
  10. Corpora used
  11. Experimental setup and metrics for evaluating results
  12. Results and analysis
  13. BERT results
  14. Features and classifier results
  15. Ensemble of BERTs
  16. ...and 4 more sections

Figures (2)

  • Figure 1: The overview of workflow of this study
  • Figure 2: Box plot of F1 scores for both corpora. The correspondence between the horizontal axis labels and datasets is as follows: A: BERTs, B: Ensemble BERTs[32, 36], C: Weighted Ensemble BERTs, D: Features & Classifiers, E: Ensemble Features & Classifiers[16-18, 41], F: Weighted Ensemble Features & Classifiers, G: Ensemble One Feature & Classifiers and BERTs [39], H: Ensemble One BERT and Features & Classifiers [40], I: Integrated Ensemble, J: Integrated Weighted Ensemble.