Sustaining Human Agency, Attending to Its Cost: An Investigation into Generative AI Design for Non-Native Speakers' Language Use

TL;DR

This study investigates how to sustain human agency in AI-mediated communication, focusing on machine translation within information-seeking conversations. It compares three human-MT interface designs—labeling, regular post-editing, and augmented post-editing—across 45 immigrant-native speaker dyads performing housing-related tasks. Findings show that post-editing interfaces increase non-native speakers' sense of agency but can reduce dyadic communication quality, whereas labeling preserves depth and alignment with moderate agency. The work offers design implications for MT and other generative AI systems, highlighting agency-cost trade-offs and the potential to customize interfaces using first-person data to better support non-native speakers.

Abstract

AI systems and tools today can generate human-like expressions on behalf of people. It raises the crucial question about how to sustain human agency in AI-mediated communication. We investigated this question in the context of machine translation (MT) assisted conversations. Our participants included 45 dyads. Each dyad consisted of one new immigrant in the United States, who leveraged MT for English information seeking as a non-native speaker, and one local native speaker, who acted as the information provider. Non-native speakers could influence the English production of their message in one of three ways: labeling the quality of MT outputs, regular post-editing without additional hints, or augmented post-editing with LLM-generated hints. Our data revealed a greater exercise of non-native speakers' agency under the two post-editing conditions. This benefit, however, came at a significant cost to the dyadic-level communication performance. We derived insights for MT and other generative AI design from our findings.

Figures (7)

• Figure 1: Composition of information in the task materials and its distribution between non-native speaking information seekers and native speaking information providers. As a participant clicks on each tab and housing options on the map view, the corresponding information will appear in a pop-up.
• Figure 2: We manipulated how non-native speakers could influence their English production in the message box. In the labeling interface condition, non-native speakers could click on the thumbs-up and thumbs-down buttons to indicate their evaluation of the translation. Their native-speaking partners received both the English message and the evaluations. In the regular post-editing interface condition, participants could edit the original MT output in the English text box. In the augmented post-editing interface condition, participants could access two paraphrases of the initial MT output and edit it in the English text box. Their native speaking partners received the final version of English messages in the two post-editing conditions.
• Figure 3: Non-native speakers' agency by the human-MT interface condition and the rater's language background
• Figure 4: Dyadic information exchange process as in the breadth (left) and depth (right) by the human-MT interface condition
• Figure 5: Unique information pieces initiated by the human-MT interface condition and individual language background