Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

It is Simple Sometimes: A Study On Improving Aspect-Based Sentiment Analysis Performance

Laura Cabello, Uchenna Akujuobi

TL;DR

PFInstruct addresses the need for simple, effective prompt-based learning for ABSA subtasks by adding an NLP-task prefix to the InstructABSA framework. The approach treats all ABSA subtasks as generative tasks and leverages RE/NER prefixes or even noisy prefixes to augment context, achieving improvements across SemEval, ERSA, and SentiHood datasets, including a $+3.28$ F1 on ATE Rest14 and a $+5.43$ F1 average on AOOE across SemEval datasets. A noisy prefix also boosts performance, indicating instruction-following benefits beyond clean prompts, though it introduces higher variance. The method shows competitive results in biomedical ERSA and generalizes to domain shifts, illustrating that a lightweight, prefix-based instruction-tuning strategy can effectively handle ABSA subtasks across diverse domains.

Abstract

Aspect-Based Sentiment Analysis (ABSA) involves extracting opinions from textual data about specific entities and their corresponding aspects through various complementary subtasks. Several prior research has focused on developing ad hoc designs of varying complexities for these subtasks. In this paper, we present a generative framework extensible to any ABSA subtask. We build upon the instruction tuned model proposed by Scaria et al. (2023), who present an instruction-based model with task descriptions followed by in-context examples on ABSA subtasks. We propose PFInstruct, an extension to this instruction learning paradigm by appending an NLP-related task prefix to the task description. This simple approach leads to improved performance across all tested SemEval subtasks, surpassing previous state-of-the-art (SOTA) on the ATE subtask (Rest14) by +3.28 F1-score, and on the AOOE subtask by an average of +5.43 F1-score across SemEval datasets. Furthermore, we explore the impact of the prefix-enhanced prompt quality on the ABSA subtasks and find that even a noisy prefix enhances model performance compared to the baseline. Our method also achieves competitive results on a biomedical domain dataset (ERSA).

It is Simple Sometimes: A Study On Improving Aspect-Based Sentiment Analysis Performance

TL;DR

PFInstruct addresses the need for simple, effective prompt-based learning for ABSA subtasks by adding an NLP-task prefix to the InstructABSA framework. The approach treats all ABSA subtasks as generative tasks and leverages RE/NER prefixes or even noisy prefixes to augment context, achieving improvements across SemEval, ERSA, and SentiHood datasets, including a F1 on ATE Rest14 and a F1 average on AOOE across SemEval datasets. A noisy prefix also boosts performance, indicating instruction-following benefits beyond clean prompts, though it introduces higher variance. The method shows competitive results in biomedical ERSA and generalizes to domain shifts, illustrating that a lightweight, prefix-based instruction-tuning strategy can effectively handle ABSA subtasks across diverse domains.

Abstract

Aspect-Based Sentiment Analysis (ABSA) involves extracting opinions from textual data about specific entities and their corresponding aspects through various complementary subtasks. Several prior research has focused on developing ad hoc designs of varying complexities for these subtasks. In this paper, we present a generative framework extensible to any ABSA subtask. We build upon the instruction tuned model proposed by Scaria et al. (2023), who present an instruction-based model with task descriptions followed by in-context examples on ABSA subtasks. We propose PFInstruct, an extension to this instruction learning paradigm by appending an NLP-related task prefix to the task description. This simple approach leads to improved performance across all tested SemEval subtasks, surpassing previous state-of-the-art (SOTA) on the ATE subtask (Rest14) by +3.28 F1-score, and on the AOOE subtask by an average of +5.43 F1-score across SemEval datasets. Furthermore, we explore the impact of the prefix-enhanced prompt quality on the ABSA subtasks and find that even a noisy prefix enhances model performance compared to the baseline. Our method also achieves competitive results on a biomedical domain dataset (ERSA).
Paper Structure (18 sections, 2 figures, 12 tables)

This paper contains 18 sections, 2 figures, 12 tables.

Table of Contents

  1. Introduction
  2. Contributions
  3. Related Work
  4. Background and Methodology
  5. Background: ABSA
  6. Methodology
  7. Results
  8. Analysis of SemEval subtasks
  9. Error Analysis
  10. Analysis of ERSA and SentiHood
  11. Error Analysis
  12. Domain generalization
  13. Discussion
  14. Conclusion
  15. Experimental details
  16. ...and 3 more sections

Figures (2)

  • Figure 1: Illustration of model input and ABSA subtasks examined in this paper. The prefix can vary between NLP-related tasks (instruction) or textual noise (random words), followed by the subtask definition, few examples and the corresponding sample input for each subtask. The model is expected to follow the instructions and generate a prediction. Subtasks belong to three distinct data sources: SemEval, ERSA and SentiHood from different domains.
  • Figure 2: Out-of-domain evaluation. F1-scores are averaged across five random initialization seeds; error bars show the standard deviation. Models are trained ('tr') on one domain and evaluated ('tst') on a distinct domain. Legends indicate the prefix prompt used. 'No' stands for no use of prefix. RE is not evaluated for ATE (see Section \ref{['sec:results']}).