Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Large-Scale Aspect-Based Sentiment Analysis with Reasoning-Infused LLMs

Paweł Liskowski, Krzysztof Jankowski

TL;DR

The paper tackles real-world aspect-based sentiment analysis by extending ABSA with reasoning-infused LLMs, scalable synthetic data, and multilingual capabilities. It introduces Arctic-ABSA, comprising decoder-only and encoder-only models, plus reasoning-injection and a novel Upside-Down Synthetic Data Generation framework, culminating in ABSA-mix, a large, cross-domain benchmark. Key results show the 395M encoder and 8B decoder achieving competitive or superior performance against large proprietary models on standard ABSA tasks, with strong multilingual generalization and improved training efficiency from reasoning pretraining. Collectively, the work offers a scalable, cross-lingual ABSA solution suitable for commercial deployment and sets a benchmark for future research with their expansive ABSA-mix dataset.

Abstract

We introduce Arctic-ABSA, a collection of powerful models for real-life aspect-based sentiment analysis (ABSA). Our models are tailored to commercial needs, trained on a large corpus of public data alongside carefully generated synthetic data, resulting in a dataset 20 times larger than SemEval14. We extend typical ABSA models by expanding the number of sentiment classes from the standard three (positive, negative, neutral) to five, adding mixed and unknown classes, while also jointly predicting overall text sentiment and supporting multiple languages. We experiment with reasoning injection by fine-tuning on Chain-of-Thought (CoT) examples and introduce a novel reasoning pretraining technique for encoder-only models that significantly improves downstream fine-tuning and generalization. Our 395M-parameter encoder and 8B-parameter decoder achieve up to 10 percentage points higher accuracy than GPT-4o and Claude 3.5 Sonnet, while setting new state-of-the-art results on the SemEval14 benchmark. A single multilingual model maintains 87-91% accuracy across six languages without degrading English performance. We release ABSA-mix, a large-scale benchmark aggregating 17 public ABSA datasets across 92 domains.

Large-Scale Aspect-Based Sentiment Analysis with Reasoning-Infused LLMs

TL;DR

The paper tackles real-world aspect-based sentiment analysis by extending ABSA with reasoning-infused LLMs, scalable synthetic data, and multilingual capabilities. It introduces Arctic-ABSA, comprising decoder-only and encoder-only models, plus reasoning-injection and a novel Upside-Down Synthetic Data Generation framework, culminating in ABSA-mix, a large, cross-domain benchmark. Key results show the 395M encoder and 8B decoder achieving competitive or superior performance against large proprietary models on standard ABSA tasks, with strong multilingual generalization and improved training efficiency from reasoning pretraining. Collectively, the work offers a scalable, cross-lingual ABSA solution suitable for commercial deployment and sets a benchmark for future research with their expansive ABSA-mix dataset.

Abstract

We introduce Arctic-ABSA, a collection of powerful models for real-life aspect-based sentiment analysis (ABSA). Our models are tailored to commercial needs, trained on a large corpus of public data alongside carefully generated synthetic data, resulting in a dataset 20 times larger than SemEval14. We extend typical ABSA models by expanding the number of sentiment classes from the standard three (positive, negative, neutral) to five, adding mixed and unknown classes, while also jointly predicting overall text sentiment and supporting multiple languages. We experiment with reasoning injection by fine-tuning on Chain-of-Thought (CoT) examples and introduce a novel reasoning pretraining technique for encoder-only models that significantly improves downstream fine-tuning and generalization. Our 395M-parameter encoder and 8B-parameter decoder achieve up to 10 percentage points higher accuracy than GPT-4o and Claude 3.5 Sonnet, while setting new state-of-the-art results on the SemEval14 benchmark. A single multilingual model maintains 87-91% accuracy across six languages without degrading English performance. We release ABSA-mix, a large-scale benchmark aggregating 17 public ABSA datasets across 92 domains.
Paper Structure (32 sections, 3 equations, 4 figures, 5 tables, 2 algorithms)

This paper contains 32 sections, 3 equations, 4 figures, 5 tables, 2 algorithms.

Table of Contents

  1. Introduction
  2. Related work
  3. Model Architectures for ABSA.
  4. Reasoning for Sentiment Analysis.
  5. Synthetic Data for Sentiment Analysis.
  6. Multilingual ABSA.
  7. Fine-tuning LLMs for ABSA
  8. Decoder-only LLMs
  9. Encoder-only LLMs
  10. Reasoning injection
  11. Decoder-only reasoning training
  12. Encoder-only thinking mode
  13. Dataset
  14. Upside-Down Synthetic Data Generation
  15. Generating Mixed-Sentiment Data
  16. ...and 17 more sections

Figures (4)

  • Figure 1: Architecture of fine-tuned models (decoder and Encoder) alongside example input and output. The decoder-only model directly outputs a JSON response for all aspects, whereas the encoder-only model requires as many forward passes as there are aspects.
  • Figure 2: Comparison of validation losses of ModernBERT fine-tuned on ABSA-mix dataset and the variant with reasoning injection pretraining. Reasoning injection significantly improves the validation loss and leads to better generalization properties.
  • Figure 3: Upside-down generation process of synthetic data. We extensively use a corpus of seed data with $46$ categories (created manually and using LLMs), $1832$ aspects, and $3463$ keywords (both generated using LLMs). The difference between aspects and keywords is the following: aspects are used as labels in predictions and are often general, whereas keywords only guide the generation process for better diversity and are much more descriptive. $18$ handcrafted styles additionally improve the diversity of the generated data.
  • Figure 4: Prompt used for evaluation of the LLMs. We specify the different possible sentiments and inject text and aspects to be analyzed by the model. Additional repetitions of the JSON format improved the instruction following capabilities repeated_prompting.