Language Portability Strategies for Open-domain Dialogue with Pre-trained Language Models from High to Low Resource Languages

TL;DR

The paper investigates linguistic portability strategies for open-domain dialogue with pre-trained language models when transferring from a high-resource source language to a simulated low-resource target language, using English to French as a testbed. It compares multiple transfer approaches, including TrainOnTarget via translation of the source data, TestOnSource with NMT-in-the-loop inference, and adapter-based adaptation (MAD-X) on BLOOM to align the model with the target language. Human evaluations in spoken dialogue settings assess perceived interaction quality across strategies, providing a direct measure of portability usability. The study highlights how leveraging NMT pipelines and adapter-based methods with a multilingual PLM can mitigate data scarcity in the target language and inform practical deployment decisions for cross-language open-domain dialogue systems.

Abstract

In this paper we propose a study of linguistic portability strategies of large pre-trained language models (PLMs) used for open-domain dialogue systems in a high-resource language for this task. In particular the target low-resource language (L_T) will be simulated with French, as it lacks of task-specific resources and allows our human evaluation, when the source language (L_S) is English. For obvious reasons, recent works using such models for open-domain dialogue are mostly developed in English. Yet building specific PLMs for each possible target language supposes collecting new datasets and is costly. For this reason, trying to leverage all existing resources (PLMs and data) in both L_S and L_T , we wish to assess the performance achievable in L_T with different approaches. The first two approaches evaluate the usage of Neural Machine Translation (NMT) at different levels: TrainOnTarget where a L_S dataset is translated before fine-tuning in L_T and TestOnSource where a L_S model is coupled with NMT modules during inference. Then, the advent of BLOOM [2], the world first open-access multilingual large PLM, allow researchers to develop new approaches aiming to leverage not only the model's full accessibility but also its multilingualism and translation abilities. In this context the task is learned in L_S first and adapted to L_T using the MAD-X Adapter architecture [16]. In the two sets of experiments models are evaluated in spoken dialogue conditions with human and the strategies can be compared in terms of perceived interaction quality.

Figures (3)

• Figure 1: If the width of the figure is less than 7.8 cm use the sidecapion command to flush the caption on the left side of the page. If the figure is positioned at the top of the page, align the sidecaption with the top of the figure -- to achieve this you simply need to use the optional argument [t] with the sidecaption command
• Figure 2: If the width of the figure is less than 7.8 cm use the sidecapion command to flush the caption on the left side of the page. If the figure is positioned at the top of the page, align the sidecaption with the top of the figure -- to achieve this you simply need to use the optional argument [t] with the sidecaption command
• Figure 3: Please write your figure caption here

Theorems & Definitions (6)

• theorem 1
• definition 1
• proof
• theorem 2
• definition 2
• proof