Multi-Scale and Multi-Objective Optimization for Cross-Lingual Aspect-Based Sentiment Analysis

TL;DR

The paper tackles cross-lingual ABSA by introducing MSMO, a framework that blends multi-scale alignment (sentence- and aspect-level) with multi-objective optimization (supervised plus consistency training) and knowledge distillation. It couples a pretrained multilingual encoder with a language discriminator for sentence-level alignment and a consistency module for aspect-level alignment, leveraging code-switching and translation-based data. Empirical results on SemEval-2016 across EN, FR, ES, NL, and RU show state-of-the-art performance on both mBERT and XLM-R backbones, with sizable gains in Spanish and benefits from distillation in multilingual settings. The work also compares with large language models, finding MSMO with multilingual distillation outperforms zero-shot and LoRA-fine-tuned LLMs on token-level ABSA, underscoring practical impact for multilingual NLP tasks.

Abstract

Aspect-based sentiment analysis (ABSA) is a sequence labeling task that has garnered growing research interest in multilingual contexts. However, recent studies lack more robust feature alignment and finer aspect-level alignment. In this paper, we propose a novel framework, Multi-Scale and Multi-Objective optimization (MSMO) for cross-lingual ABSA. During multi-scale alignment, we achieve cross-lingual sentence-level and aspect-level alignment, aligning features of aspect terms in different contextual environments. Specifically, we introduce code-switched bilingual sentences into the language discriminator and consistency training modules to enhance the model's robustness. During multi-objective optimization, we design two optimization objectives: supervised training and consistency training, aiming to enhance cross-lingual semantic alignment. To further improve model performance, we incorporate distilled knowledge of the target language into the model. Results show that MSMO significantly enhances cross-lingual ABSA by achieving state-of-the-art performance across multiple languages and models.

Figures (7)

• Figure 1: An example of a cross-lingual ABSA task. We train on the source language and perform aspect term extraction and sentiment polarity prediction on the target language.
• Figure 2: The MSMO framework. It mainly comprises two basic steps: (1). Sentence-level alignment by adversarial training (§\ref{['sec:step1']}); (2). Aspect-level alignment with multi-objective optimization (§\ref{['sec:step2']}). The Pretrained Multi-Lingual Encoder connects both steps by updating the parameters from the loss of language discriminator in step 1 and from the combined loss in step 2.
• Figure 3: An example of the code-switched dataset.
• Figure 4: The single-teacher and multi-teacher distillation process.
• Figure 5: The multilingual distillation process.