From Fake Focus to Real Precision: Confusion-Driven Adversarial Attention Learning in Transformers

TL;DR

The paper tackles suboptimal token-level attention in Transformer-based sentiment analysis by introducing Adversarial Feedback for Attention (AFA), a dynamic, annotation-free mechanism that uses a Target Model and a Discriminator in a masking-based adversarial loop guided by policy-gradient optimization. AFA learns context-aware attention distributions $a$ without external labels, improving both predictive performance and interpretability. Empirical results across AG News, IMDB, SST-2, and additional tasks show state-of-the-art accuracy and strong robustness, with large-language-model experiments reporting substantial gains (notably ~12.6% for some configurations). The approach offers a practical, generalizable pathway to more reliable sentiment analysis on both lightweight models and large-scale LLMs, while maintaining a lightweight discriminator and flexible implementation.

Abstract

Transformer-based models have been widely adopted for sentiment analysis tasks due to their exceptional ability to capture contextual information. However, these methods often exhibit suboptimal accuracy in certain scenarios. By analyzing their attention distributions, we observe that existing models tend to allocate attention primarily to common words, overlooking less popular yet highly task-relevant terms, which significantly impairs overall performance. To address this issue, we propose an Adversarial Feedback for Attention(AFA) training mechanism that enables the model to automatically redistribute attention weights to appropriate focal points without requiring manual annotations. This mechanism incorporates a dynamic masking strategy that attempts to mask various words to deceive a discriminator, while the discriminator strives to detect significant differences induced by these masks. Additionally, leveraging the sensitivity of Transformer models to token-level perturbations, we employ a policy gradient approach to optimize attention distributions, which facilitates efficient and rapid convergence. Experiments on three public datasets demonstrate that our method achieves state-of-the-art results. Furthermore, applying this training mechanism to enhance attention in large language models yields a further performance improvement of 12.6%

Figures (5)

• Figure 1: An overview of AFA.The Target Model generates attention distributions over tokens and selects critical tokens for masking to confuse the Discriminator. The Discriminator is trained to distinguish between original and masked sequences, while adversarial feedback and classification loss jointly guide the Target Model to explore new attention distributions that focus on truly important tokens.
• Figure 2: Illustration of how the Target Model explores new attention distributions by performing token masking. When unimportant tokens are masked, the Discriminator can easily detect the modification. In contrast, when critical tokens are masked, the resulting masked sequence may flip the semantic label, thereby confusing the Discriminator and providing adversarial feedback to guide the Target Model.
• Figure 3: Performance degradation under token deletion on the AGNews dataset. Tokens were ranked by attention weights, and the top-N tokens were sequentially removed.
• Figure 4: Case study on the AG News dataset. The color intensity indicates attention values: darker colors correspond to higher attention weights.
• Figure 5: Sensitivity analysis with respect to the number of selected tokens $K$ on AGNews, SST-2, and IMDB datasets, and highlighting the adaptability of AFA to diverse task characteristics.