A Global-Local Attention Mechanism for Relation Classification
Yiping Sun
TL;DR
The paper tackles relation classification by reducing noise in global attention through a global-local mechanism that simultaneously attends to all words and to a focused subset of keywords. It introduces hard and soft localization to identify candidate keywords for the local channel and fuses global and local signals with a mixture weight, yielding a model that uses $s = \sum_i \alpha_i H_i$ and $\alpha_i = \gamma \alpha_{gi} + (1-\gamma) \alpha_{li}$. The approach, implemented on a BiGRU encoder, achieves state-of-the-art performance on SemEval-2010 Task 8, with soft localization reaching $85.0\%$ macro F1 and hard localization $84.7\%$, and a balanced gamma around $0.5$ providing the best results. The work demonstrates that localized keyword emphasis can improve both accuracy and interpretability of attention, with practical implications for knowledge-graph–driven NLP and retrieval tasks.
Abstract
Relation classification, a crucial component of relation extraction, involves identifying connections between two entities. Previous studies have predominantly focused on integrating the attention mechanism into relation classification at a global scale, overlooking the importance of the local context. To address this gap, this paper introduces a novel global-local attention mechanism for relation classification, which enhances global attention with a localized focus. Additionally, we propose innovative hard and soft localization mechanisms to identify potential keywords for local attention. By incorporating both hard and soft localization strategies, our approach offers a more nuanced and comprehensive understanding of the contextual cues that contribute to effective relation classification. Our experimental results on the SemEval-2010 Task 8 dataset highlight the superior performance of our method compared to previous attention-based approaches in relation classification.