Extending Czech Aspect-Based Sentiment Analysis with Opinion Terms: Dataset and LLM Benchmarks

TL;DR

A novel Czech dataset in the restaurant domain for aspect-based sentiment analysis (ABSA), enriched with annotations of opinion terms, and a proposed translation-alignment approach that offers a scalable solution for adapting ABSA resources to other low-resource languages.

Abstract

This paper introduces a novel Czech dataset in the restaurant domain for aspect-based sentiment analysis (ABSA), enriched with annotations of opinion terms. The dataset supports three distinct ABSA tasks involving opinion terms, accommodating varying levels of complexity. Leveraging this dataset, we conduct extensive experiments using modern Transformer-based models, including large language models (LLMs), in monolingual, cross-lingual, and multilingual settings. To address cross-lingual challenges, we propose a translation and label alignment methodology leveraging LLMs, which yields consistent improvements. Our results highlight the strengths and limitations of state-of-the-art models, especially when handling the linguistic intricacies of low-resource languages like Czech. A detailed error analysis reveals key challenges, including the detection of subtle opinion terms and nuanced sentiment expressions. The dataset establishes a new benchmark for Czech ABSA, and our proposed translation-alignment approach offers a scalable solution for adapting ABSA resources to other low-resource languages.

Figures (5)

• Figure 1: Example of the original annotations (top) and the updated versions after our modifications (bottom).
• Figure 2: Example of converting ABSA annotations into output sequences for sequence-to-sequence models. Special tokens represent aspect terms, opinion terms, categories, sentiment polarities, and tuple separators.
• Figure 3: Prompt for the ACOS task with example input, expected output (green box), and one demonstration (dashed box, used only in few-shot scenarios).
• Figure 4: Prompt for translating the ACOS dataset from English to Czech with aligned labels. The full prompt contains five different representative input/output examples.
• Figure 5: Number of error types for each dataset for GPT-4o mini in zero-shot settings, LLaMA 3.3 70B in few-shot settings, and fine-tuned mT5.