T-T: Table Transformer for Tagging-based Aspect Sentiment Triplet Extraction

TL;DR

The paper tackles ASTE by reframing table tagging with a dedicated Table-Transformer (T-T) that introduces stripe attention to curb the $O(n^4)$ cost of full attention on 2D table sequences and a loop-shift strategy to enable cross-block information flow. The method encodes the sentence into a relation table via biaffine interactions, then refines it with T-T to capture rich word relations before decoding TL/BR coordinates and sentiments for triplets. Empirical results across four SemEval-derived datasets show state-of-the-art or near-state-of-the-art performance with lower computational cost compared to prior table-tagging methods, and ablations confirm the importance of both stripe attention and loop-shift. The findings demonstrate that transformer-based relation encoding, when adapted to 2D table structures, yields robust ASTE performance while maintaining efficiency, with potential applicability to related information-extraction tasks.

Abstract

Aspect sentiment triplet extraction (ASTE) aims to extract triplets composed of aspect terms, opinion terms, and sentiment polarities from given sentences. The table tagging method is a popular approach to addressing this task, which encodes a sentence into a 2-dimensional table, allowing for the tagging of relations between any two words. Previous efforts have focused on designing various downstream relation learning modules to better capture interactions between tokens in the table, revealing that a stronger capability to capture relations can lead to greater improvements in the model. Motivated by this, we attempt to directly utilize transformer layers as downstream relation learning modules. Due to the powerful semantic modeling capability of transformers, it is foreseeable that this will lead to excellent improvement. However, owing to the quadratic relation between the length of the table and the length of the input sentence sequence, using transformers directly faces two challenges: overly long table sequences and unfair local attention interaction. To address these challenges, we propose a novel Table-Transformer (T-T) for the tagging-based ASTE method. Specifically, we introduce a stripe attention mechanism with a loop-shift strategy to tackle these challenges. The former modifies the global attention mechanism to only attend to a 2-dimensional local attention window, while the latter facilitates interaction between different attention windows. Extensive and comprehensive experiments demonstrate that the T-T, as a downstream relation learning module, achieves state-of-the-art performance with lower computational costs.

Figures (5)

• Figure 1: A toy example of three different ASTE methods. Our T-T model achieves commendable results in both performance and cost.
• Figure 2: a) shows the architecture of our table tagging model for ASTE. One of the core components is the configurable relation encoding module. TL and BR donate the top-left and bottom-right vertex cells of the sentiment region, respectively. b) shows the architecture of our proposed T-T module. In sub-figure c), the left half shows the query matrix and key matrix divided into $4^2$ blocks, while the right half represents the attention map, where the colored blocks indicate the dot product computations. In sub-figure d), the final outputs of different layers shift between state (i) and state (ii). For a boundary token (marked with a red star), it attends to tokens 1, 2, and 4 in the output state (i), and to tokens 5, 7, and 8 in the output state (ii).
• Figure 3: The sensitivity of different hyperparameters.
• Figure 4: F1 scores for different aspect-opinion word distances on the test set. The sample counts in different distance intervals are 2442, 449, 107, and 52, respectively.
• Figure 5: Performance of different word spans. Single. denotes triplets with single-word aspects and opinions. Multi. A./Multi. O. denote triplets with multiple-word aspects/opinions.