Entity-Aware Self-Attention and Contextualized GCN for Enhanced Relation Extraction in Long Sentences

TL;DR

A novel model, Entity-aware Self-attention Contextualized GCN (ESC-GCN), which efficiently incorporates syntactic structure of input sentences and semantic context of sequences and excels in extracting relations between entities of long sentences is proposed.

Abstract

Relation extraction as an important natural Language processing (NLP) task is to identify relations between named entities in text. Recently, graph convolutional networks over dependency trees have been widely used to capture syntactic features and achieved attractive performance. However, most existing dependency-based approaches ignore the positive influence of the words outside the dependency trees, sometimes conveying rich and useful information on relation extraction. In this paper, we propose a novel model, Entity-aware Self-attention Contextualized GCN (ESC-GCN), which efficiently incorporates syntactic structure of input sentences and semantic context of sequences. To be specific, relative position self-attention obtains the overall semantic pairwise correlation related to word position, and contextualized graph convolutional networks capture rich intra-sentence dependencies between words by adequately pruning operations. Furthermore, entity-aware attention layer dynamically selects which token is more decisive to make final relation prediction. In this way, our proposed model not only reduces the noisy impact from dependency trees, but also obtains easily-ignored entity-related semantic representation. Extensive experiments on various tasks demonstrate that our model achieves encouraging performance as compared to existing dependency-based and sequence-based models. Specially, our model excels in extracting relations between entities of long sentences.

Figures (7)

• Figure 1: Example of dependency parsing for two sentences expressing a ternary interaction. The SDP between three entities (identify with different color) is highlighted in bold (edges and tokens). The root node of the LCA subtree of entities is present. The dotted edges indicate tokens $K$=1 away from the subtree. Tokens "partial response" are off these two paths.
• Figure 2: Overview of our proposed ESC-GCN model, an example sentence "The deletion mutation on exon-19 of EFGR gene".
• Figure 3: Relative Position self-attention structure.
• Figure 4: Visualization of attention scores in the relative position self-attention layer. Darker color indicates higher score.
• Figure 5: Test set performance with different sentence lengths for ESC-GCN, C-GCN and PA-LSTM. The PA-LSTM results are from zhang2017position, and the C-GCN are from zhang2018graph.