Multiple Sources are Better Than One: Incorporating External Knowledge in Low-Resource Glossing

TL;DR

This paper tackles data scarcity in automatic glossing for low-resource languages by integrating multiple external knowledge sources: sentence- and token-level translations, external dictionaries, and LLM prompting as a post-correction step. Building on a strong baseline, it introduces a translation encoder and a character-based decoder, and uses in-context learning with carefully selected prompts to refine lexical morphemes without requiring fine-tuning. Empirical results on the SIGMORPHON 2023 data show average improvements around 5 percentage points, with Gitksan achieving about 10-point gains, and even larger benefits in ultra-low-resource settings with fewer than 100 glossed sentences. The findings demonstrate the practical impact of combining translations, dictionaries, and LLM-based post-editing for scalable, accurate documentation of diverse, data-sparse languages.

Abstract

In this paper, we address the data scarcity problem in automatic data-driven glossing for low-resource languages by coordinating multiple sources of linguistic expertise. We supplement models with translations at both the token and sentence level as well as leverage the extensive linguistic capability of modern LLMs. Our enhancements lead to an average absolute improvement of 5%-points in word-level accuracy over the previous state of the art on a typologically diverse dataset spanning six low-resource languages. The improvements are particularly noticeable for the lowest-resourced language Gitksan, where we achieve a 10%-point improvement. Furthermore, in a simulated ultra-low resource setting for the same six languages, training on fewer than 100 glossed sentences, we establish an average 10%-point improvement in word-level accuracy over the previous state-of-the-art system.

Figures (12)

• Figure 1: When glossing input such as the French sentence Le chien aboie, our system utilizes multiple information sources: an English sentence-level translation, general linguistic knowledge provided by an LLM and dictionary definitions for the input tokens.
• Figure 2: Pipeline of girrbach-2023-tu's model.
• Figure 3: Pipeline of the proposed work. The lower portion of the diagram demonstrates how attention weights inform the model when predicting the glossing targets.
• Figure 4: The procedure of selecting in-context learning examples to generate components for LLM prompting.
• Figure 5: Lexical morpheme and word-level accuracy on Arapaho. We incorporate prompting with the encoder-decoder model which is enriched with translation.