Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Massively Multilingual Joint Segmentation and Glossing

Michael Ginn, Lindia Tjuatja, Enora Rice, Ali Marashian, Maria Valentini, Jasmine Xu, Graham Neubig, Alexis Palmer

TL;DR

This work tackles the interpretability and trust barriers of automated glossing by presenting PolyGloss, a multilingual model that jointly predicts morphological segmentation and interlinear glosses. The authors extend and standardize the GlossLM corpus, introduce three task formats with a principled interleaved approach, and demonstrate state-of-the-art glossing performance alongside strong segmentation and alignment across many languages. They also show that per-language perplexity strongly predicts glossing quality, enabling practical safeguards, and propose efficient adaptation via LoRA for new data. The model is open-source and designed for integration into real-world annotation workflows, offering a scalable, adaptable solution for language documentation at scale.

Abstract

Automated interlinear gloss prediction with neural networks is a promising approach to accelerate language documentation efforts. However, while state-of-the-art models like GlossLM achieve high scores on glossing benchmarks, user studies with linguists have found critical barriers to the usefulness of such models in real-world scenarios. In particular, existing models typically generate morpheme-level glosses but assign them to whole words without predicting the actual morpheme boundaries, making the predictions less interpretable and thus untrustworthy to human annotators. We conduct the first study on neural models that jointly predict interlinear glosses and the corresponding morphological segmentation from raw text. We run experiments to determine the optimal way to train models that balance segmentation and glossing accuracy, as well as the alignment between the two tasks. We extend the training corpus of GlossLM and pretrain PolyGloss, a family of seq2seq multilingual models for joint segmentation and glossing that outperforms GlossLM on glossing and beats various open-source LLMs on segmentation, glossing, and alignment. In addition, we demonstrate that PolyGloss can be quickly adapted to a new dataset via low-rank adaptation.

Massively Multilingual Joint Segmentation and Glossing

TL;DR

This work tackles the interpretability and trust barriers of automated glossing by presenting PolyGloss, a multilingual model that jointly predicts morphological segmentation and interlinear glosses. The authors extend and standardize the GlossLM corpus, introduce three task formats with a principled interleaved approach, and demonstrate state-of-the-art glossing performance alongside strong segmentation and alignment across many languages. They also show that per-language perplexity strongly predicts glossing quality, enabling practical safeguards, and propose efficient adaptation via LoRA for new data. The model is open-source and designed for integration into real-world annotation workflows, offering a scalable, adaptable solution for language documentation at scale.

Abstract

Automated interlinear gloss prediction with neural networks is a promising approach to accelerate language documentation efforts. However, while state-of-the-art models like GlossLM achieve high scores on glossing benchmarks, user studies with linguists have found critical barriers to the usefulness of such models in real-world scenarios. In particular, existing models typically generate morpheme-level glosses but assign them to whole words without predicting the actual morpheme boundaries, making the predictions less interpretable and thus untrustworthy to human annotators. We conduct the first study on neural models that jointly predict interlinear glosses and the corresponding morphological segmentation from raw text. We run experiments to determine the optimal way to train models that balance segmentation and glossing accuracy, as well as the alignment between the two tasks. We extend the training corpus of GlossLM and pretrain PolyGloss, a family of seq2seq multilingual models for joint segmentation and glossing that outperforms GlossLM on glossing and beats various open-source LLMs on segmentation, glossing, and alignment. In addition, we demonstrate that PolyGloss can be quickly adapted to a new dataset via low-rank adaptation.
Paper Structure (37 sections, 8 figures, 9 tables)

This paper contains 37 sections, 8 figures, 9 tables.

Table of Contents

  1. Introduction
  2. Corpus
  3. Evaluation
  4. Glossing
  5. Segmentation
  6. Alignment
  7. Model
  8. Task Format
  9. Multitask Prediction
  10. Concatenated Prediction
  11. Interleaved Prediction
  12. Base Model
  13. Training
  14. Baselines
  15. Multilingual Baselines
  16. ...and 22 more sections

Figures (8)

  • Figure 1: An interlinear glossed text example, showing the Arapaho for "Are you hungry, Andy?". Our model predicts the segmentation and gloss line from the transcribed text.
  • Figure 2: Relationship between validation set perplexity for a given language and glossing performance, as measured by morpheme error rate. There is a strong correlation ($r^2=0.951$), indicating that perplexity can be used as a rough predictor of glossing performance.
  • Figure 3: Scores for monolingual models using various approaches to multitask training. A lower glossing MER is better; higher is better for the other two metrics. Scores are averaged across nine languages and reported with standard error.
  • Figure 4: Ablation comparing monolingual joint models (using the interleaved format) and the multilingual PolyGloss using the same format. A lower glossing MER is better; higher is better for the other two metrics. Scores are averaged across nine languages and reported with standard error.
  • Figure 5: Scores for PolyGloss multilingual models using three different data formats. Scores are averaged across nine languages and reported with standard error.
  • ...and 3 more figures