NushuRescue: Revitalization of the Endangered Nushu Language with AI

TL;DR

Endangered language Nüshu faces critical data scarcity; the authors propose NüshuRescue, a LLM-in-the-loop data augmentation framework to build and scale corpora with minimal human input. They contribute NCGold, a $500$-sentence Nüshu-Chinese parallel corpus, and NCSilver, $98$ generated translations, demonstrating a model-agnostic approach that can work with GPT-4-Turbo. GPT-4-Turbo achieved $48.69\%$ translation accuracy on $50$ withheld sentences using only $35$ seed examples, and the authors provide FastText and Seq2Seq baselines for Nüshu modeling and MT. This framework lowers human labor in endangered language revitalization and is adaptable to other low-resource languages.

Abstract

The preservation and revitalization of endangered and extinct languages is a meaningful endeavor, conserving cultural heritage while enriching fields like linguistics and anthropology. However, these languages are typically low-resource, making their reconstruction labor-intensive and costly. This challenge is exemplified by Nushu, a rare script historically used by Yao women in China for self-expression within a patriarchal society. To address this challenge, we introduce NushuRescue, an AI-driven framework designed to train large language models (LLMs) on endangered languages with minimal data. NushuRescue automates evaluation and expands target corpora to accelerate linguistic revitalization. As a foundational component, we developed NCGold, a 500-sentence Nushu-Chinese parallel corpus, the first publicly available dataset of its kind. Leveraging GPT-4-Turbo, with no prior exposure to Nushu and only 35 short examples from NCGold, NushuRescue achieved 48.69% translation accuracy on 50 withheld sentences and generated NCSilver, a set of 98 newly translated modern Chinese sentences of varying lengths. A sample of both NCGold and NCSilver is included in the Supplementary Materials. Additionally, we developed FastText-based and Seq2Seq models to further support research on Nushu. NushuRescue provides a versatile and scalable tool for the revitalization of endangered languages, minimizing the need for extensive human input.

Figures (14)

• Figure 1: A simplified, stylized rendition of the NüshuRescue generation framework, using samples from NCGold.
• Figure 2: Comparison of mapped Chinese (logographic) and Nüshu (syllabic) characters for the word "ocean".
• Figure 3: Nüshu-Chinese one-to-many and many-to-one mapping; in extreme cases, one Nüshu word can map up to 35 Chinese words.
• Figure 4: Manual Data Collection: Step-by-step transcription process.
• Figure 5: A small sample of newly mapped words not in the official Nüshu-Chinese dictionary; an original contribution.