In-context Language Learning for Endangered Languages in Speech Recognition

TL;DR

Endangered-language ASR faces data scarcity; this paper investigates in-context language learning (ICLL) to enable LLMs to learn unseen languages from hundreds of samples. It introduces sample retrieval strategies and LLM-based hypothesis re-ranking to perform ICLL for language modeling and ASR, showing example-specific retrieval and probability-based hypothesis scoring yield the strongest gains. Across four endangered languages, ICLL achieves perplexity reductions and competitive or superior ASR performance against corpus-trained baselines, while preserving the LLM's original capabilities and using modest compute. The work provides public code and demonstrates a practical path to extending ASR coverage to low-resource languages with limited data and compute.

Abstract

With approximately 7,000 languages spoken worldwide, current large language models (LLMs) support only a small subset. Prior research indicates LLMs can learn new languages for certain tasks without supervised data. We extend this investigation to speech recognition, investigating whether LLMs can learn unseen, low-resource languages through in-context learning (ICL). With experiments on four diverse endangered languages that LLMs have not been trained on, we find that providing more relevant text samples enhances performance in both language modelling and Automatic Speech Recognition (ASR) tasks. Furthermore, we show that the probability-based approach outperforms the traditional instruction-based approach in language learning. Lastly, we show ICL enables LLMs to achieve ASR performance that is comparable to or even surpasses dedicated language models trained specifically for these languages, while preserving the original capabilities of the LLMs. Our code is publicly available.