Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

New Textual Corpora for Serbian Language Modeling

Mihailo Škorić, Nikola Janković

TL;DR

The paper tackles the scarcity and uneven quality of Serbian and Serbo-Croatian textual resources for language model pre-training. It surveys publicly available corpora, classifies them by origin and form, and introduces three new curated corpora: Umbrella corp, S.T.A.R.S., and PaSaž, aiming to balance coverage and quality. A word-frequency-based evaluation demonstrates the distinctiveness of the new corpora, notably SrpELTeC and S.T.A.R.S., and shows the value of deduplication and aggregation in expanding usable data. By providing publicly accessible resources and a framework for assessing corpus uniqueness, the work supports more robust Serbian LM pre-training and cross-lingual Serbo-Croatian research.

Abstract

This paper will present textual corpora for Serbian (and Serbo-Croatian), usable for the training of large language models and publicly available at one of the several notable online repositories. Each corpus will be classified using multiple methods and its characteristics will be detailed. Additionally, the paper will introduce three new corpora: a new umbrella web corpus of Serbo-Croatian, a new high-quality corpus based on the doctoral dissertations stored within National Repository of Doctoral Dissertations from all Universities in Serbia, and a parallel corpus of abstract translation from the same source. The uniqueness of both old and new corpora will be accessed via frequency-based stylometric methods, and the results will be briefly discussed.

New Textual Corpora for Serbian Language Modeling

TL;DR

The paper tackles the scarcity and uneven quality of Serbian and Serbo-Croatian textual resources for language model pre-training. It surveys publicly available corpora, classifies them by origin and form, and introduces three new curated corpora: Umbrella corp, S.T.A.R.S., and PaSaž, aiming to balance coverage and quality. A word-frequency-based evaluation demonstrates the distinctiveness of the new corpora, notably SrpELTeC and S.T.A.R.S., and shows the value of deduplication and aggregation in expanding usable data. By providing publicly accessible resources and a framework for assessing corpus uniqueness, the work supports more robust Serbian LM pre-training and cross-lingual Serbo-Croatian research.

Abstract

This paper will present textual corpora for Serbian (and Serbo-Croatian), usable for the training of large language models and publicly available at one of the several notable online repositories. Each corpus will be classified using multiple methods and its characteristics will be detailed. Additionally, the paper will introduce three new corpora: a new umbrella web corpus of Serbo-Croatian, a new high-quality corpus based on the doctoral dissertations stored within National Repository of Doctoral Dissertations from all Universities in Serbia, and a parallel corpus of abstract translation from the same source. The uniqueness of both old and new corpora will be accessed via frequency-based stylometric methods, and the results will be briefly discussed.
Paper Structure (11 sections, 2 figures, 8 tables)

This paper contains 11 sections, 2 figures, 8 tables.

Table of Contents

  1. Introduction
  2. Available Corpora
  3. Publicly available plain-text corpora
  4. Publicly available annotated corpora
  5. Publicly available parallel corpora
  6. New Corpora
  7. Aggregated web corpus - Umbrella corp
  8. Set of theses and academic research in Serbian - S.T.A.R.S.
  9. Parallel abstracts corpus - PaSaž
  10. Evaluation
  11. Conclusion

Figures (2)

  • Figure 1: Aggregation hierarchy of publicly available Serbo-Croatian plain-text corpora. Blue color represents Serbian corpora, cyan Montenegrin, pink Croatian, lime Bosnian and the color purple represents corpora with mixed language origin.
  • Figure 2: Corpus uniqueness visualized through a two-dimensional network graph representation of the calculated corpus-similarity matrix, where the edges represent distances between each corpus and the colors represent a clustering spectrum.