Enhancing Low-Resource Relation Representations through Multi-View Decoupling

TL;DR

This work tackles relation extraction under low-resource settings by proposing MVRE, a prompt-based framework that decouples each relation into multiple views to enrich the latent representation. It introduces a sampling strategy for multi-view relation latent spaces and two training aids: a Global-Local Loss to align semantic aspects across views and Dynamic Initialization to seed trainable virtual relation words. MVRE demonstrates state-of-the-art performance on SemEval, TACRED, and TACREV in 1-, 5-, and 16-shot scenarios, with ablations confirming the value of multi-view decoupling, GL, and DI. The approach enhances robustness of prompt-tuning for RE in scarce data regimes and provides insights into multi-view representation learning with virtual words. Its practical impact lies in enabling more reliable RE in low-resource domains and guiding future work on view-based representation learning in PLMs.

Abstract

Recently, prompt-tuning with pre-trained language models (PLMs) has demonstrated the significantly enhancing ability of relation extraction (RE) tasks. However, in low-resource scenarios, where the available training data is scarce, previous prompt-based methods may still perform poorly for prompt-based representation learning due to a superficial understanding of the relation. To this end, we highlight the importance of learning high-quality relation representation in low-resource scenarios for RE, and propose a novel prompt-based relation representation method, named MVRE (\underline{M}ulti-\underline{V}iew \underline{R}elation \underline{E}xtraction), to better leverage the capacity of PLMs to improve the performance of RE within the low-resource prompt-tuning paradigm. Specifically, MVRE decouples each relation into different perspectives to encompass multi-view relation representations for maximizing the likelihood during relation inference. Furthermore, we also design a Global-Local loss and a Dynamic-Initialization method for better alignment of the multi-view relation-representing virtual words, containing the semantics of relation labels during the optimization learning process and initialization. Extensive experiments on three benchmark datasets show that our method can achieve state-of-the-art in low-resource settings.

Figures (5)

• Figure 1: (a) An example of prompt-tuning for RE. Red-colored words indicate the subject, while blue-colored words indicate the object. (b) The concept of multi-view decoupling attempts to encompass various aspects of a relation using multiple relation representations.
• Figure 2: (a) An illustrative comparison of the relation latent space learning process between MVRE and previous prompt-based works. We employ multi-view relation representations to cover a broader latent space in low-resource scenarios. (b) The MVRE framework incorporates Multi-view Decoupling Learning, Global-Local Loss and Dynamic Initialization processes.
• Figure 3: Effect of the number of [MASK] on MVRE.
• Figure 4: MVRE under low-resource conditions vs. MVRE with only one [MASK] under more resource-rich conditions.
• Figure 5: A heat map between different virtual words and aspects. Each row shows how virtual words relate to different views.