Multilinguality in LLM-Designed Reward Functions for Restless Bandits: Effects on Task Performance and Fairness
Ambreesh Parthasarathy, Chandrasekar Subramanian, Ganesh Senrayan, Shreyash Adappanavar, Aparna Taneja, Balaraman Ravindran, Milind Tambe
TL;DR
This work investigates how prompting an LLM to design reward functions for Restless Multi-Armed Bandits (RMABs) from multilingual prompts affects task performance and fairness. Using the DLM framework with prompts in English, Hindi, Tamil, and a low-resource language (Tulu) in a synthetic public-health-like RMAB environment, the study evaluates reward-function quality, task alignment, and demographic parity across age groups. Key findings show English prompts outperform others in both reward proposals and allocations, while phrasing and explicit goal statements significantly influence performance; increasing prompt complexity consistently degrades performance, with English being more robust. The study also reveals that lower-resource languages and more complex prompts are more prone to unfairness across unintended features, highlighting the need for prompt design safeguards, language-aware improvements, and guardrails for equitable deployments.
Abstract
Restless Multi-Armed Bandits (RMABs) have been successfully applied to resource allocation problems in a variety of settings, including public health. With the rapid development of powerful large language models (LLMs), they are increasingly used to design reward functions to better match human preferences. Recent work has shown that LLMs can be used to tailor automated allocation decisions to community needs using language prompts. However, this has been studied primarily for English prompts and with a focus on task performance only. This can be an issue since grassroots workers, especially in developing countries like India, prefer to work in local languages, some of which are low-resource. Further, given the nature of the problem, biases along population groups unintended by the user are also undesirable. In this work, we study the effects on both task performance and fairness when the DLM algorithm, a recent work on using LLMs to design reward functions for RMABs, is prompted with non-English language commands. Specifically, we run the model on a synthetic environment for various prompts translated into multiple languages. The prompts themselves vary in complexity. Our results show that the LLM-proposed reward functions are significantly better when prompted in English compared to other languages. We also find that the exact phrasing of the prompt impacts task performance. Further, as prompt complexity increases, performance worsens for all languages; however, it is more robust with English prompts than with lower-resource languages. On the fairness side, we find that low-resource languages and more complex prompts are both highly likely to create unfairness along unintended dimensions.