PGSO: Prompt-based Generative Sequence Optimization Network for Aspect-based Sentiment Analysis

TL;DR

This work targets the ABSA task by addressing the inadequacy of standard position embeddings in generative PLMs to capture long-distance aspect–opinion relations. It introduces two sequence optimization strategies—rule-based static and score-based dynamic—and unifies them in the Prompt-based Generative Sequence Optimization Network (PGSO). PGSO combines a prompt construction module that reframes the task as a cloze-style problem with a sequence regulator that enriches semantic/syntactic representations via a GAT-based syntax encoder and a score-based re-ranking mechanism. Empirical results on 12 datasets across four ABSA tasks show state-of-the-art F1 performance, with notable improvements in long-distance relation extraction and solid ablations confirming the value of dynamic sequence regulation and syntactic guidance. The approach offers a practical, unified, and scalable enhancement for ABSA with generative models, reducing reliance on task-specific templates and improving cross-task transferability.

Abstract

Recently, generative pre-training based models have demonstrated remarkable results on Aspect-based Sentiment Analysis (ABSA) task. However, previous works overemphasize crafting various templates to paraphrase training targets for enhanced decoding, ignoring the internal optimizations on generative models. Despite notable results achieved by these target-oriented optimization methods, they struggle with the complicated long texts since the implicit long-distance relation, e.g., aspect-opinion relation, is difficult to extract under the position embedding mechanism in generative models. Thus, in this paper, we first clarify the causes of the problem and introduce two sequence optimization strategies: the rule-based static optimization and the score-based dynamic optimization. The rule-based approach relies on handcraft priority of dependency relation to reorder the context, while the score-based algorithm dynamically regulates the contextual sequence by calculating word position scores using neural network. Based on the dynamic optimization structure, we further propose a unified Prompt-based Generative Sequence Optimization network (named PGSO), which jointly optimizes the training target as well as the generative model. Specifically, PGSO contains two components, namely, prompt construction and sequence regulator. The former constructs a task-specific prompt based on unsupervised training objects to fully utilize the pre-trained model. The latter jointly leverages semantic, syntactic and original-sequence information to dynamically regulate contextual sequence. Our experiments conducted on four ABSA tasks across multiple benchmarks indicate that PGSO outperforms state-of-the-art methods, with an average improvement of 3.52% in F1 score.

Figures (5)

• Figure 1: Part of the relative position embedding mechanism. The initial distance between the words pizza and good is measured at 15. This distance is subsequently reduced to 9 through a compression process. As a result, the precise positional information provided by the input text is diminished. Notably, with larger distances, the bucket or range is also becomes wider.
• Figure 2: Structure of the rule-based static optimization method.
• Figure 3: Structure of the score-based dynamic optimization method.
• Figure 4: Overall architecture of PGSO. The architecture of the Prompt-based Generative Sequence Optimization (PGSO) model extends beyond the conventional encoder-decoder framework of the T5 model to incorporate two distinct components. Prompt Construction: This component is designed to narrow the gap between pre-training task and downstream ABSA task, maximizing the utilization of our proposed model. It is composed of two specialized prompts: a semantic prompt and a few-shot prompt. Sequence Regulator: This module includes a syntax encoder and a score calculator. The syntax encoder leverages rich syntax information to enhance the textual representations, thereby enhancing the model's interpretative ability. The score calculator operates on the refined representations to obtain the position score to each word in the input text. Subsequently, it produces the optimized sequence, which is meticulously ordered based on the computed scores, thereby ensuring that the output is not only syntactically coherent but also semantically rich and contextually relevant.
• Figure 5: Comparisons of GAS-T5, PGSO (w/o SR) and PGSO with respect to the performace across various distance between the aspect and opinion. Notation "SR" represents Sequence Regulator.