Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Building Foundations for Natural Language Processing of Historical Turkish: Resources and Models

Şaziye Betül Özateş, Tarık Emre Tıraş, Ece Elif Adak, Berat Doğan, Fatih Burak Karagöz, Efe Eren Genç, Esma F. Bilgin Taşdemir

TL;DR

The paper addresses the lack of NLP resources for historical Turkish by introducing three foundational resources (HisTR NER dataset, OTA-BOUN UD treebank, and the Ottoman Text Corpus OTC) and providing transformer-based baselines for NER, parsing, and POS tagging. It demonstrates that Turkish-specific PLMs like BERTurk, when fine-tuned on historical data, outperform multilingual models, though substantial domain adaptation challenges remain across historical periods. All resources and models are publicly released to establish benchmarks and encourage continual pretraining on OTC for improved period-aware NLP of Ottoman Turkish. The work lays a concrete foundation for subsequent advances in historical Turkish NLP and domain-specific evaluation.

Abstract

This paper introduces foundational resources and models for natural language processing (NLP) of historical Turkish, a domain that has remained underexplored in computational linguistics. We present the first named entity recognition (NER) dataset, HisTR and the first Universal Dependencies treebank, OTA-BOUN for a historical form of the Turkish language along with transformer-based models trained using these datasets for named entity recognition, dependency parsing, and part-of-speech tagging tasks. Additionally, we introduce Ottoman Text Corpus (OTC), a clean corpus of transliterated historical Turkish texts that spans a wide range of historical periods. Our experimental results show significant improvements in the computational analysis of historical Turkish, achieving promising results in tasks that require understanding of historical linguistic structures. They also highlight existing challenges, such as domain adaptation and language variations across time periods. All of the presented resources and models are made available at https://huggingface.co/bucolin to serve as a benchmark for future progress in historical Turkish NLP.

Building Foundations for Natural Language Processing of Historical Turkish: Resources and Models

TL;DR

The paper addresses the lack of NLP resources for historical Turkish by introducing three foundational resources (HisTR NER dataset, OTA-BOUN UD treebank, and the Ottoman Text Corpus OTC) and providing transformer-based baselines for NER, parsing, and POS tagging. It demonstrates that Turkish-specific PLMs like BERTurk, when fine-tuned on historical data, outperform multilingual models, though substantial domain adaptation challenges remain across historical periods. All resources and models are publicly released to establish benchmarks and encourage continual pretraining on OTC for improved period-aware NLP of Ottoman Turkish. The work lays a concrete foundation for subsequent advances in historical Turkish NLP and domain-specific evaluation.

Abstract

This paper introduces foundational resources and models for natural language processing (NLP) of historical Turkish, a domain that has remained underexplored in computational linguistics. We present the first named entity recognition (NER) dataset, HisTR and the first Universal Dependencies treebank, OTA-BOUN for a historical form of the Turkish language along with transformer-based models trained using these datasets for named entity recognition, dependency parsing, and part-of-speech tagging tasks. Additionally, we introduce Ottoman Text Corpus (OTC), a clean corpus of transliterated historical Turkish texts that spans a wide range of historical periods. Our experimental results show significant improvements in the computational analysis of historical Turkish, achieving promising results in tasks that require understanding of historical linguistic structures. They also highlight existing challenges, such as domain adaptation and language variations across time periods. All of the presented resources and models are made available at https://huggingface.co/bucolin to serve as a benchmark for future progress in historical Turkish NLP.
Paper Structure (26 sections, 4 figures, 7 tables)

This paper contains 26 sections, 4 figures, 7 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Resources for Historical Turkish NLP
  4. HisTR: A Historical Turkish NER Dataset
  5. OTA-BOUN: A Universal Dependencies Treebank for Historical Turkish
  6. Annotation Scheme
  7. Treebank Statistics
  8. Challenges in the Syntactic Annotation
  9. Ottoman Text Corpus: A Clean Text Corpus of Historical Turkish Documents
  10. Challenges in Pre-processing
  11. Tasks and Models
  12. Named Entity Recognition
  13. Dependency Parsing and Part-of-Speech Tagging
  14. Pre-trained Language Models
  15. Model Details
  16. ...and 11 more sections

Figures (4)

  • Figure 1: Transcription of an excerpt from the original document which is written with the Perso-Arabic script.
  • Figure 2: CoNLL-U Representation of an example sentence from our OTA-BOUN historical Turkish treebank.
  • Figure 3: t-SNE visualization of documents in the Ottoman Text Corpus: Each point represents a document, color-coded by topic. This visualization highlights thematic clusters within the corpus and shows how topics are distributed periodically, complementing the map’s representation of vocabulary diversity and topic.
  • Figure 4: Dependency tree representations of a historical Turkish sentence (above) and its rewritten version in modern Turkish (below). The highlighted portions enclosed in colored circles indicate corresponding segments in the sentences. English translations of words are provided in italic within parentheses. Words of a sentence that do not exist in the other sentence are underlined in the figure. English translation of the sentence: "The late Damat İbrahim Paşa managed to develop Muşkara, his birthplace, and turn it into a town, and changed its name to Nevşehir."