Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Semantic Alignment across Ancient Egyptian Language Stages via Normalization-Aware Multitask Learning

He Huang

Abstract

We study word-level semantic alignment across four historical stages of Ancient Egyptian. These stages differ in script and orthography, and parallel data are scarce. We jointly train a compact encoder-decoder model with a shared byte-level tokenizer on all four stages, combining masked language modeling (MLM), translation language modeling (TLM), sequence-to-sequence translation, and part-of-speech tagging under a task-aware loss with fixed weights and uncertainty-based scaling. To reduce surface divergence we add Latin transliteration and IPA reconstruction as auxiliary views. We integrate these views through KL-based consistency and through embedding-level fusion. We evaluate alignment quality using pairwise metrics, specifically ROC-AUC and triplet accuracy, on curated Egyptian-English and intra-Egyptian cognate datasets. Translation yields the strongest gains. IPA with KL consistency improves cross-branch alignment, while early fusion demonstrates limited efficacy. Although the overall alignment remains limited, the findings provide a reproducible baseline and practical guidance for modeling historical languages under real constraints. They also show how normalization and task design shape what counts as alignment in typologically distant settings.

Semantic Alignment across Ancient Egyptian Language Stages via Normalization-Aware Multitask Learning

Abstract

We study word-level semantic alignment across four historical stages of Ancient Egyptian. These stages differ in script and orthography, and parallel data are scarce. We jointly train a compact encoder-decoder model with a shared byte-level tokenizer on all four stages, combining masked language modeling (MLM), translation language modeling (TLM), sequence-to-sequence translation, and part-of-speech tagging under a task-aware loss with fixed weights and uncertainty-based scaling. To reduce surface divergence we add Latin transliteration and IPA reconstruction as auxiliary views. We integrate these views through KL-based consistency and through embedding-level fusion. We evaluate alignment quality using pairwise metrics, specifically ROC-AUC and triplet accuracy, on curated Egyptian-English and intra-Egyptian cognate datasets. Translation yields the strongest gains. IPA with KL consistency improves cross-branch alignment, while early fusion demonstrates limited efficacy. Although the overall alignment remains limited, the findings provide a reproducible baseline and practical guidance for modeling historical languages under real constraints. They also show how normalization and task design shape what counts as alignment in typologically distant settings.
Paper Structure (47 sections, 7 equations, 5 figures, 2 tables)

This paper contains 47 sections, 7 equations, 5 figures, 2 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. NLP for Egyptian and Coptic.
  4. Alignment objectives and multitask learning.
  5. Normalization for historical texts.
  6. Methods
  7. Dataset
  8. Model
  9. Baseline and Task-Level Comparisons
  10. Representation-Level Normalization Strategies
  11. View Integration Methods
  12. (A) Consistency via KL Divergence.
  13. (B) Early Fusion via Embedding Mixture.
  14. Strategy Comparison.
  15. Experiments
  16. ...and 32 more sections

Figures (5)

  • Figure 1: Overview of the experimental pipeline. The ablation study isolates two independent dimensions: (1) task-level supervision, without normalization; and (2) representation-level inputs under a standalone MLM task.
  • Figure 2: t-SNE visualization of word embeddings across four Egyptian language stages through the MLM-only baseline and the full multitask model (MLM+TLM+Translation+POS).
  • Figure 3: t-SNE visualization including English as a pivot language through the MLM-only baseline with or without Latin normalization and KL consistency training.
  • Figure 4: Alignment performance (AUC and Accuracy) between Ancient Egyptian and English under different multitask and normalization settings. Each cell displays AUC (top) and Accuracy (bottom, in parentheses), rounded to two decimal places. The colormap encodes AUC scores from 0.40 to 1.00, which serves as the primary metric; Accuracy is shown for reference. The highest AUC in each row is highlighted with a white rectangle.
  • Figure 5: Intra-Egyptian alignment performance (AUC and Accuracy) across historical stages and training variants. The highest AUC in each row is highlighted with a white rectangle.