Are Multilingual Language Models an Off-ramp for Under-resourced Languages? Will we arrive at Digital Language Equality in Europe in 2030?
Georg Rehm, Annika Grützner-Zahn, Fabio Barth
TL;DR
The paper addresses digital language inequality in Europe and investigates whether multilingual LLMs can serve as an off-ramp for under-resourced languages. It synthesizes policy context (ELE, DLE) and data-space infrastructure (Common European Language Data Space, LDS) with model-centric evidence, including cross-lingual transfer capabilities of European multilingual LLMs. It analyzes data availability, training efficiency techniques, and evaluation biases, highlighting both the potential of multilingual LLMs and the open research questions necessary for practical deployment. The authors advocate a pragmatic, phased path toward digital language equality by 2030, leveraging off-ramp technologies in tandem with robust data spaces and data unlocking efforts, while outlining key questions about data needs, language diversity, and base-language choices.
Abstract
Large language models (LLMs) demonstrate unprecedented capabilities and define the state of the art for almost all natural language processing (NLP) tasks and also for essentially all Language Technology (LT) applications. LLMs can only be trained for languages for which a sufficient amount of pre-training data is available, effectively excluding many languages that are typically characterised as under-resourced. However, there is both circumstantial and empirical evidence that multilingual LLMs, which have been trained using data sets that cover multiple languages (including under-resourced ones), do exhibit strong capabilities for some of these under-resourced languages. Eventually, this approach may have the potential to be a technological off-ramp for those under-resourced languages for which "native" LLMs, and LLM-based technologies, cannot be developed due to a lack of training data. This paper, which concentrates on European languages, examines this idea, analyses the current situation in terms of technology support and summarises related work. The article concludes by focusing on the key open questions that need to be answered for the approach to be put into practice in a systematic way.