Reward-based Input Construction for Cross-document Relation Extraction

TL;DR

This work proposes REward-based Input Construction (REIC), the first learning-based sentence selector for cross-document RE, and trains REIC using reinforcement learning with RE prediction scores as rewards, demonstrating the superiority of the method over heuristic methods for different RE structures and backbones in cross-document RE.

Abstract

Relation extraction (RE) is a fundamental task in natural language processing, aiming to identify relations between target entities in text. While many RE methods are designed for a single sentence or document, cross-document RE has emerged to address relations across multiple long documents. Given the nature of long documents in cross-document RE, extracting document embeddings is challenging due to the length constraints of pre-trained language models. Therefore, we propose REward-based Input Construction (REIC), the first learning-based sentence selector for cross-document RE. REIC extracts sentences based on relational evidence, enabling the RE module to effectively infer relations. Since supervision of evidence sentences is generally unavailable, we train REIC using reinforcement learning with RE prediction scores as rewards. Experimental results demonstrate the superiority of our method over heuristic methods for different RE structures and backbones in cross-document RE. Our code is publicly available at https://github.com/aailabkaist/REIC.

Figures (9)

• Figure 1: An illustrated comparison between Snippet and selected sentences using our REward-based Input Construction (REIC) for cross-document relation extraction. The figure depicts an example triplet (Kubuntu, x86-64, platform) with the text path ('Mir (software)', 'X86-64'), including three bridge entities (Ubuntu, Linux, Intel) abbreviated as (Bridge1, Bridge2, Bridge3). Dash and solid arrows signify the selection process of Snippet and REIC, respectively, while gradient-colored arrows indicate connections between the head and tail entities. REIC selects important sentences from any position within a path to determine the relation between the head and tail entity, whereas Snippet only includes sentences located around the head or tail entity.
• Figure 2: Overview of our cross-document RE framework. Solid black arrows represent forward propagation, while the dashed red arrows represent the loss signal. Documents from each text path are processed by the sentence selection module to construct the input for the RE module, which then obtains relation predictions.
• Figure 3: Illustration of the REIC module: BERT encoder, policy network, and LSTM. Each sentence $s_m$, combined with the target sentence $s_{\text{tgt}}$, is processed by the fixed BERT to obtain the embedding $z_m$. Then, these embeddings pass through the policy network $G_{\boldsymbol{\phi}}$ to obtain the selection probability $\hat{\pi}_{m,t}:=\hat{\pi}(s_m|S_t)$, and the sentence is sampled with this probability. The embedding of the selected sentence is fed to the LSTM to incorporate information from previously selected sentences into the policy network. In this way, sentences are sequentially selected, with masking applied to embeddings for subsequent selections, as indicated by the dotted line.
• Figure 4: Experimental results on F1s based on the average number of tokens in a bag of text paths with ECRIM (BERT) RE module.
• Figure 5: Experimental results on F1s based on the average number of bridge entities in a bag of text paths with ECRIM (BERT) RE module.