Efficient Information Extraction in Few-Shot Relation Classification through Contrastive Representation Learning

TL;DR

A novel approach to enhance information extraction combining multiple sentence representations and contrastive learning is introduced, validating the adaptability of the approach, maintaining robust performance in scenarios that include relation descriptions, and showcasing its flexibility to adapt to different resource constraints.

Abstract

Differentiating relationships between entity pairs with limited labeled instances poses a significant challenge in few-shot relation classification. Representations of textual data extract rich information spanning the domain, entities, and relations. In this paper, we introduce a novel approach to enhance information extraction combining multiple sentence representations and contrastive learning. While representations in relation classification are commonly extracted using entity marker tokens, we argue that substantial information within the internal model representations remains untapped. To address this, we propose aligning multiple sentence representations, such as the [CLS] token, the [MASK] token used in prompting, and entity marker tokens. Our method employs contrastive learning to extract complementary discriminative information from these individual representations. This is particularly relevant in low-resource settings where information is scarce. Leveraging multiple sentence representations is especially effective in distilling discriminative information for relation classification when additional information, like relation descriptions, are not available. We validate the adaptability of our approach, maintaining robust performance in scenarios that include relation descriptions, and showcasing its flexibility to adapt to different resource constraints.

Figures (4)

• Figure 1: Overview of the MultiRep model, which integrates relation description information. The $\circ$ represents the vector dot product between the prototypes and the query samples, while the addition operation is denoted by $\otimes$. Attracting and repelling forces in contrastive learning are represented by $\rightarrow \leftarrow$ and $\dashleftarrow \dashrightarrow$, respectively. and illustrate the representations extracted from input sentences and relation descriptions, respectively.
• Figure 2: Average accuracy and standard deviation for varying number of representations $M$ evaluated on the FewRel validation set.
• Figure 3: t-SNE visualization of 120 randomly sampled support embeddings for hard relation classification examples obtained from MultiRep.
• Figure 4: t-SNE visualization of 120 randomly sampled support embeddings for hard relation classification examples obtained from MultiRep w/o CL.