Structured Semantic Information Helps Retrieve Better Examples for In-Context Learning in Few-Shot Relation Extraction

TL;DR

The paper tackles FSRE under a $5$-way $1$-shot in-context learning setting by automatically collecting additional demonstrations. It introduces a hybrid demonstration-selection framework that combines LLM-generated paraphrases/new examples with retrieval from unannotated corpora using structured lexico-syntactic rules via SoftMatcher and FAISS, promoting diversity while preserving relational faithfulness. Empirical results show that semantically-grounded retrieval often outperforms SBERT-based retrieval and LLM paraphrase alone, and that the hybrid strategy achieves state-of-the-art performance on FS-TACRED and strong gains on FS-FewRel, especially for smaller LLMs, with transfer across datasets and model families. The approach reduces reliance on extensive labeled data while enabling robust cross-domain FSRE, and qualitative analyses highlight the value of diversified, relation-faithful demonstrations balanced with similarity to the gold example.

Abstract

This paper presents several strategies to automatically obtain additional examples for in-context learning of one-shot relation extraction. Specifically, we introduce a novel strategy for example selection, in which new examples are selected based on the similarity of their underlying syntactic-semantic structure to the provided one-shot example. We show that this method results in complementary word choices and sentence structures when compared to LLM-generated examples. When these strategies are combined, the resulting hybrid system achieves a more holistic picture of the relations of interest than either method alone. Our framework transfers well across datasets (FS-TACRED and FS-FewRel) and LLM families (Qwen and Gemma). Overall, our hybrid selection method consistently outperforms alternative strategies and achieves state-of-the-art performance on FS-TACRED and strong gains on a customized FewRel subset.

Figures (1)

• Figure 1: Our approach to in-context learning for 1-shot relation extraction. Our method complements the (gold) 1-shot ("Support sentence" in the figure) with additional examples generated using shallower LLM-based methods (i.e., paraphrasing the gold example, or generating new examples based on the relation definition) or deeper methods using the underlying syntactic-semantic representations of candidate examples. When combining these strategies, the resulting few-shot relation extraction system achieves better results across two corpora, across several small language models.