Embedded Translations for Low-resource Automated Glossing

TL;DR

A hard-attentional neural model is augmented with embedded translation information extracted from interlinear glossed text, which highlights the critical role of translation information in boosting the system's performance, especially in processing and interpreting modest data sources.

Abstract

We investigate automatic interlinear glossing in low-resource settings. We augment a hard-attentional neural model with embedded translation information extracted from interlinear glossed text. After encoding these translations using large language models, specifically BERT and T5, we introduce a character-level decoder for generating glossed output. Aided by these enhancements, our model demonstrates an average improvement of 3.97\%-points over the previous state of the art on datasets from the SIGMORPHON 2023 Shared Task on Interlinear Glossing. In a simulated ultra low-resource setting, trained on as few as 100 sentences, our system achieves an average 9.78\%-point improvement over the plain hard-attentional baseline. These results highlight the critical role of translation information in boosting the system's performance, especially in processing and interpreting modest data sources. Our findings suggest a promising avenue for the documentation and preservation of languages, with our experiments on shared task datasets indicating significant advancements over the existing state of the art.

Figures (8)

• Figure 1: Pipeline of the proposed work. The lower portion of the diagram demonstrates how attention weights inform the model when predicting the glossing targets.
• Figure 2: Difference from mean attention weights of glossed output tokens (y-axis) with respect to encoded translation tokens (x-axis) for a Natügu example (attention weights are derived from the model BERT+attn+chr).
• Figure 3: Difference from mean attention weights of glossed output tokens (y-axis) with respect to encoded translation tokens (x-axis) for an Arapaho example (attention weights are derived from the model BERT+attn+chr (left) and the model LSTM+attm (right)). The gold-standard glosses for this sentence: IC.it.is-2S IC.be.had.as.father.by.all-2S.
• Figure 4: Difference from mean attention weights of glossed output tokens (y-axis) with respect to encoded translation tokens (x-axis) for a Gitksan example (attention weights are derived from the model BERT+attn+chr (left) and the model LSTM+attm (right)). The gold-standard glosses for this sentence: CCNJ want-3.II PROSP-3.I tell-T-3.II OBL-1PL.II MANR LVB-3.II.
• Figure 5: Difference from mean attention weights of glossed output tokens (y-axis) with respect to encoded translation tokens (x-axis) for a Lezgi example (attention weights are derived from the model BERT+attn+chr (left) and the model LSTM+attm (right)). The gold-standard glosses for this sentence: 1pl.abs return-AOR this one there village-ERG-DAT.