Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Chain-of-Translation Prompting (CoTR): A Novel Prompting Technique for Low Resource Languages

Tejas Deshpande, Nidhi Kowtal, Raviraj Joshi

TL;DR

This paper introduces Chain-of-Translation Prompting (CoTR), a prompting strategy that translates input text from a low-resource language (Marathi) into a high-resource language (English), performs the target NLP task on the translated text, and optionally translates the output back, all within a single prompt. Evaluated across multiple models (including GPT-4o variants, Llama-3, and Gemma-9B) and tasks (sentiment analysis, hate speech detection, subject classification, and headline generation) using Marathi datasets (MahaSent, MahaHate, MahaNews-SHC, XLSum), CoTR consistently improves performance over direct Marathi prompting, with the largest gains in hate speech and sentiment tasks and when using smaller models. The study highlights that LLM-based translations often exceed baseline Google Translate performance, supporting the viability of translation-based prompting for low-resource languages and suggesting avenues for future integration with Chain-of-Thought prompting. The findings have practical implications for expanding multilingual NLP capabilities and generating high-quality synthetic data for underrepresented languages.

Abstract

This paper introduces Chain of Translation Prompting (CoTR), a novel strategy designed to enhance the performance of language models in low-resource languages. CoTR restructures prompts to first translate the input context from a low-resource language into a higher-resource language, such as English. The specified task like generation, classification, or any other NLP function is then performed on the translated text, with the option to translate the output back to the original language if needed. All these steps are specified in a single prompt. We demonstrate the effectiveness of this method through a case study on the low-resource Indic language Marathi. The CoTR strategy is applied to various tasks, including sentiment analysis, hate speech classification, subject classification and text generation, and its efficacy is showcased by comparing it with regular prompting methods. Our results underscore the potential of translation-based prompting strategies to significantly improve multilingual LLM performance in low-resource languages, offering valuable insights for future research and applications. We specifically see the highest accuracy improvements with the hate speech detection task. The technique also has the potential to enhance the quality of synthetic data generation for underrepresented languages using LLMs.

Chain-of-Translation Prompting (CoTR): A Novel Prompting Technique for Low Resource Languages

TL;DR

This paper introduces Chain-of-Translation Prompting (CoTR), a prompting strategy that translates input text from a low-resource language (Marathi) into a high-resource language (English), performs the target NLP task on the translated text, and optionally translates the output back, all within a single prompt. Evaluated across multiple models (including GPT-4o variants, Llama-3, and Gemma-9B) and tasks (sentiment analysis, hate speech detection, subject classification, and headline generation) using Marathi datasets (MahaSent, MahaHate, MahaNews-SHC, XLSum), CoTR consistently improves performance over direct Marathi prompting, with the largest gains in hate speech and sentiment tasks and when using smaller models. The study highlights that LLM-based translations often exceed baseline Google Translate performance, supporting the viability of translation-based prompting for low-resource languages and suggesting avenues for future integration with Chain-of-Thought prompting. The findings have practical implications for expanding multilingual NLP capabilities and generating high-quality synthetic data for underrepresented languages.

Abstract

This paper introduces Chain of Translation Prompting (CoTR), a novel strategy designed to enhance the performance of language models in low-resource languages. CoTR restructures prompts to first translate the input context from a low-resource language into a higher-resource language, such as English. The specified task like generation, classification, or any other NLP function is then performed on the translated text, with the option to translate the output back to the original language if needed. All these steps are specified in a single prompt. We demonstrate the effectiveness of this method through a case study on the low-resource Indic language Marathi. The CoTR strategy is applied to various tasks, including sentiment analysis, hate speech classification, subject classification and text generation, and its efficacy is showcased by comparing it with regular prompting methods. Our results underscore the potential of translation-based prompting strategies to significantly improve multilingual LLM performance in low-resource languages, offering valuable insights for future research and applications. We specifically see the highest accuracy improvements with the hate speech detection task. The technique also has the potential to enhance the quality of synthetic data generation for underrepresented languages using LLMs.
Paper Structure (10 sections, 5 figures, 4 tables)

This paper contains 10 sections, 5 figures, 4 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Methodology
  4. Chain of Translation Prompting
  5. Datasets Used
  6. Evaluation Methodology
  7. Results and Discussion
  8. Classification Task
  9. Generation Task
  10. Future Work and Conclusion

Figures (5)

  • Figure 1: A brief overview of the Chain of Translation Prompting (CoTR) for an annotation task. The technique modifies the input prompt to encapsulate the translation of the non-English input context to English, followed by performing the target task on the translated text.
  • Figure 2: Prompt for Classification Task
  • Figure 3: Prompt for Generation Task
  • Figure 4: Classification Task using Chain of Translation Prompting
  • Figure 5: Generative Task using Chain of Translation Prompting