Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Responsible Multilingual Large Language Models: A Survey of Development, Applications, and Societal Impact

Junhua Liu, Bin Fu

TL;DR

This work provides essential guidance for practitioners and researchers working to develop more inclusive and effective multilingual AI systems through real-world applications in customer service, search engines, and machine translation.

Abstract

Multilingual Large Language Models (MLLMs) represent a pivotal advancement in democratizing artificial intelligence across linguistic boundaries. While theoretical foundations are well-established, practical implementation guidelines remain scattered. This work bridges this gap by providing a comprehensive end-to-end framework for developing and deploying MLLMs in production environments. We make three distinctive contributions: First, we present an actionable pipeline from data pre-processing through deployment, integrating insights from academic research and industrial applications. Second, using Llama2 as a case study, we provide detailed optimization strategies for enhancing multilingual capabilities, including curriculum learning approaches for balancing high-resource and low-resource languages, tokenization strategies, and effective sampling methods. Third, we offer an interdisciplinary analysis that considers technical, linguistic, and cultural perspectives in MLLM development. Our findings reveal critical challenges in supporting linguistic diversity, with 88.38% of world languages categorized as low-resource, affecting over a billion speakers. We examine practical solutions through real-world applications in customer service, search engines, and machine translation. By synthesizing theoretical frameworks with production-ready implementation strategies, this survey provides essential guidance for practitioners and researchers working to develop more inclusive and effective multilingual AI systems.

Responsible Multilingual Large Language Models: A Survey of Development, Applications, and Societal Impact

TL;DR

This work provides essential guidance for practitioners and researchers working to develop more inclusive and effective multilingual AI systems through real-world applications in customer service, search engines, and machine translation.

Abstract

Multilingual Large Language Models (MLLMs) represent a pivotal advancement in democratizing artificial intelligence across linguistic boundaries. While theoretical foundations are well-established, practical implementation guidelines remain scattered. This work bridges this gap by providing a comprehensive end-to-end framework for developing and deploying MLLMs in production environments. We make three distinctive contributions: First, we present an actionable pipeline from data pre-processing through deployment, integrating insights from academic research and industrial applications. Second, using Llama2 as a case study, we provide detailed optimization strategies for enhancing multilingual capabilities, including curriculum learning approaches for balancing high-resource and low-resource languages, tokenization strategies, and effective sampling methods. Third, we offer an interdisciplinary analysis that considers technical, linguistic, and cultural perspectives in MLLM development. Our findings reveal critical challenges in supporting linguistic diversity, with 88.38% of world languages categorized as low-resource, affecting over a billion speakers. We examine practical solutions through real-world applications in customer service, search engines, and machine translation. By synthesizing theoretical frameworks with production-ready implementation strategies, this survey provides essential guidance for practitioners and researchers working to develop more inclusive and effective multilingual AI systems.

Paper Structure

This paper contains 49 sections, 2 equations, 17 figures, 5 tables.

Table of Contents

  1. Introduction
  2. Contributions
  3. MLLM Production Framework
  4. Optimization Strategies
  5. Interdisciplinary Considerations
  6. Overview of Multilingual Research
  7. Multilingual Research
  8. Social Aspect
  9. Linguistic Aspect
  10. Cultural and Ethical Aspect
  11. Model Aspect
  12. Challenges in Multilingual NLP
  13. Current Trends
  14. Challenges
  15. Multilingual Models
  16. ...and 34 more sections

Figures (17)

  • Figure 1: Different categories of language resources in NLP systems
  • Figure 2: Evaluation Benchmarks and Related Datasets for MLLMs
  • Figure 3: Evaluation methods for MLLMs
  • Figure 4: Pre-training data processing flow for MLLMs
  • Figure 5: Subword Fertility Analysis: Definition and Cross-linguistic Comparison (a) Example calculation of subword fertility ratio (b) Subword fertility rates for different languages including Arabic (AR), English (EN), Finnish (FI), Indonesian (ID), Japanese (JA), Korean (KO), Russian (RU), Turkish (TR), and Chinese (ZH)
  • ...and 12 more figures