Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

AraPoemBERT: A Pretrained Language Model for Arabic Poetry Analysis

Faisal Qarah

TL;DR

The proposed AraPoemBERT model significantly outperformed previous work and other comparative models in the tasks of poems' sentiment analysis, achieving an accuracy of 78.95\%, and poetry meter classification, while significantly expanding the scope of these two problems.

Abstract

Arabic poetry, with its rich linguistic features and profound cultural significance, presents a unique challenge to the Natural Language Processing (NLP) field. The complexity of its structure and context necessitates advanced computational models for accurate analysis. In this paper, we introduce AraPoemBERT, an Arabic language model pretrained exclusively on Arabic poetry text. To demonstrate the effectiveness of the proposed model, we compared AraPoemBERT with 5 different Arabic language models on various NLP tasks related to Arabic poetry. The new model outperformed all other models and achieved state-of-the-art results in most of the downstream tasks. AraPoemBERT achieved unprecedented accuracy in two out of three novel tasks: poet's gender classification (99.34\% accuracy), and poetry sub-meter classification (97.79\% accuracy). In addition, the model achieved an accuracy score in poems' rhyme classification (97.73\% accuracy) which is almost equivalent to the best score reported in this study. Moreover, the proposed model significantly outperformed previous work and other comparative models in the tasks of poems' sentiment analysis, achieving an accuracy of 78.95\%, and poetry meter classification (99.03\% accuracy), while significantly expanding the scope of these two problems. The dataset used in this study, contains more than 2.09 million verses collected from online sources, each associated with various attributes such as meter, sub-meter, poet, rhyme, and topic. The results demonstrate the effectiveness of the proposed model in understanding and analyzing Arabic poetry, achieving state-of-the-art results in several tasks and outperforming previous works and other language models included in the study. AraPoemBERT model is publicly available on \url{https://huggingface.co/faisalq}.

AraPoemBERT: A Pretrained Language Model for Arabic Poetry Analysis

TL;DR

The proposed AraPoemBERT model significantly outperformed previous work and other comparative models in the tasks of poems' sentiment analysis, achieving an accuracy of 78.95\%, and poetry meter classification, while significantly expanding the scope of these two problems.

Abstract

Arabic poetry, with its rich linguistic features and profound cultural significance, presents a unique challenge to the Natural Language Processing (NLP) field. The complexity of its structure and context necessitates advanced computational models for accurate analysis. In this paper, we introduce AraPoemBERT, an Arabic language model pretrained exclusively on Arabic poetry text. To demonstrate the effectiveness of the proposed model, we compared AraPoemBERT with 5 different Arabic language models on various NLP tasks related to Arabic poetry. The new model outperformed all other models and achieved state-of-the-art results in most of the downstream tasks. AraPoemBERT achieved unprecedented accuracy in two out of three novel tasks: poet's gender classification (99.34\% accuracy), and poetry sub-meter classification (97.79\% accuracy). In addition, the model achieved an accuracy score in poems' rhyme classification (97.73\% accuracy) which is almost equivalent to the best score reported in this study. Moreover, the proposed model significantly outperformed previous work and other comparative models in the tasks of poems' sentiment analysis, achieving an accuracy of 78.95\%, and poetry meter classification (99.03\% accuracy), while significantly expanding the scope of these two problems. The dataset used in this study, contains more than 2.09 million verses collected from online sources, each associated with various attributes such as meter, sub-meter, poet, rhyme, and topic. The results demonstrate the effectiveness of the proposed model in understanding and analyzing Arabic poetry, achieving state-of-the-art results in several tasks and outperforming previous works and other language models included in the study. AraPoemBERT model is publicly available on \url{https://huggingface.co/faisalq}.
Paper Structure (21 sections, 5 equations, 11 figures, 14 tables)

This paper contains 21 sections, 5 equations, 11 figures, 14 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Arabic Poetry Analysis
  4. Arabic Language Models
  5. Transformers
  6. Proposed Model: AraPoemBERT
  7. Model Architecture
  8. Dataset
  9. Experiments and Results
  10. Text Preprocessing
  11. Model Pretraining
  12. Downstream Tasks
  13. Sentiment Analysis
  14. Poetry Meters
  15. Poetry Sub-Meters
  16. ...and 6 more sections

Figures (11)

  • Figure 1: Overview of the transformer model architecture vaswani2017attention
  • Figure 2: An example of masked language model (MLM) training objective
  • Figure 3: An illustration of BERT pretraining process devlin2018bert.
  • Figure 4: The distribution of number of tokens per verse in AraPoems dataset
  • Figure 5: A sample of AraPoems dataset
  • ...and 6 more figures