Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

The Model Arena for Cross-lingual Sentiment Analysis: A Comparative Study in the Era of Large Language Models

Xiliang Zhu, Shayna Gardiner, Tere Roldán, David Rossouw

TL;DR

This work undertakes an empirical analysis to compare the cross-lingual transfer capability of public Small Multilingual Language Models like XLM-R, against English-centric LLMs such as Llama-3, in the context of sentiment analysis across English, Spanish, French and Chinese.

Abstract

Sentiment analysis serves as a pivotal component in Natural Language Processing (NLP). Advancements in multilingual pre-trained models such as XLM-R and mT5 have contributed to the increasing interest in cross-lingual sentiment analysis. The recent emergence in Large Language Models (LLM) has significantly advanced general NLP tasks, however, the capability of such LLMs in cross-lingual sentiment analysis has not been fully studied. This work undertakes an empirical analysis to compare the cross-lingual transfer capability of public Small Multilingual Language Models (SMLM) like XLM-R, against English-centric LLMs such as Llama-3, in the context of sentiment analysis across English, Spanish, French and Chinese. Our findings reveal that among public models, SMLMs exhibit superior zero-shot cross-lingual performance relative to LLMs. However, in few-shot cross-lingual settings, public LLMs demonstrate an enhanced adaptive potential. In addition, we observe that proprietary GPT-3.5 and GPT-4 lead in zero-shot cross-lingual capability, but are outpaced by public models in few-shot scenarios.

The Model Arena for Cross-lingual Sentiment Analysis: A Comparative Study in the Era of Large Language Models

TL;DR

This work undertakes an empirical analysis to compare the cross-lingual transfer capability of public Small Multilingual Language Models like XLM-R, against English-centric LLMs such as Llama-3, in the context of sentiment analysis across English, Spanish, French and Chinese.

Abstract

Sentiment analysis serves as a pivotal component in Natural Language Processing (NLP). Advancements in multilingual pre-trained models such as XLM-R and mT5 have contributed to the increasing interest in cross-lingual sentiment analysis. The recent emergence in Large Language Models (LLM) has significantly advanced general NLP tasks, however, the capability of such LLMs in cross-lingual sentiment analysis has not been fully studied. This work undertakes an empirical analysis to compare the cross-lingual transfer capability of public Small Multilingual Language Models (SMLM) like XLM-R, against English-centric LLMs such as Llama-3, in the context of sentiment analysis across English, Spanish, French and Chinese. Our findings reveal that among public models, SMLMs exhibit superior zero-shot cross-lingual performance relative to LLMs. However, in few-shot cross-lingual settings, public LLMs demonstrate an enhanced adaptive potential. In addition, we observe that proprietary GPT-3.5 and GPT-4 lead in zero-shot cross-lingual capability, but are outpaced by public models in few-shot scenarios.
Paper Structure (21 sections, 2 figures, 6 tables)

This paper contains 21 sections, 2 figures, 6 tables.

Table of Contents

  1. Introduction
  2. Background
  3. Cross-lingual Sentiment Analysis
  4. Sentiment Analysis in Conversational Transcripts
  5. Methodology
  6. Supervised Fine-tuning
  7. In-context Learning
  8. Experiment
  9. Dataset
  10. Selected pre-trained Models
  11. Experiment Setup
  12. Results
  13. Zero-shot Cross-lingual Transfer
  14. Few-shot Cross-lingual Transfer
  15. Conclusion
  16. ...and 6 more sections

Figures (2)

  • Figure 1: Diagram of zero- and few- shot cross-lingual sentiment analysis from English (EN) to French (FR) under Supervised Fine-tuning (left) and In-context learning (right).
  • Figure 2: Average F1 score performance comparison (across ES, FR and ZH) under N-shot settings. GPT-3.5 is not included in this 600-shot due to the context length limit.