TPN: Transferable Proto-Learning Network towards Few-shot Document-Level Relation Extraction

TL;DR

A Transferable Proto-Learning Network (TPN) is introduced to address the challenging issue of few-shot document-level relation extraction, and achieves competitive performance with approximately half the parameter size.

Abstract

Few-shot document-level relation extraction suffers from poor performance due to the challenging cross-domain transferability of NOTA (none-of-the-above) relation representation. In this paper, we introduce a Transferable Proto-Learning Network (TPN) to address the challenging issue. It comprises three core components: Hybrid Encoder hierarchically encodes semantic content of input text combined with attention information to enhance the relation representations. As a plug-and-play module for Out-of-Domain (OOD) Detection, Transferable Proto-Learner computes NOTA prototype through an adaptive learnable block, effectively mitigating NOTA bias across various domains. Dynamic Weighting Calibrator detects relation-specific classification confidence, serving as dynamic weights to calibrate the NOTA-dominant loss function. Finally, to bolster the model's cross-domain performance, we complement it with virtual adversarial training (VAT). We conduct extensive experimental analyses on FREDo and ReFREDo, demonstrating the superiority of TPN. Compared to state-of-the-art methods, our approach achieves competitive performance with approximately half the parameter size. Data and code are available at https://github.com/EchoDreamer/TPN.

Figures (7)

• Figure 1: Illustration of relation prototype embedding space for previous global multi-proto (left) and our TPN (right). The previous methods share the embedding of the NOTA prototype across different domains. Instead, our TPN dynamically adapts the NOTA prototype from Domain$_{train}$ to Domain$_{test}$.
• Figure 2: The overall architecture of our proposed TPN framework.
• Figure 3: Visualization of extracted prototypes for the support set from validation dataset.
• Figure 4: Visualization of extracted prototypes for the support set from in-domain testing dataset.
• Figure 5: Effect of hyperparameter $\alpha$ and $\beta$ under the 3-Doc task setting in ReFREDo.