Unlocking Instructive In-Context Learning with Tabular Prompting for Relational Triple Extraction

TL;DR

This work tackles prompting design and demonstration selection in in-context learning for relational triple extraction by introducing TableIE, a tabular prompting method that frames RTE as table generation, paired with I$^2$CL, which selectively annotates a small, semantically informed set of unlabeled samples. A triple-level similarity metric based on Pompeiu–Hausdorff distance guides a learned retriever to assemble effective demonstrations, and three annotation strategies balance content coverage with budget constraints. Empirical results on CoNLL04 and NYT show that TableIE consistently outperforms TextIE and CodeIE, while I$^2$CL with balanced or coverage-based selection yields state-of-the-art few-shot performance across GPT-3, GPT-4, and ChatGPT backbones, with favorable cost-efficiency. The approach highlights the importance of structured output formats and semantic-aware sample selection for scalable, zero-/low-resource RTE in real-world applications.

Abstract

The in-context learning (ICL) for relational triple extraction (RTE) has achieved promising performance, but still encounters two key challenges: (1) how to design effective prompts and (2) how to select proper demonstrations. Existing methods, however, fail to address these challenges appropriately. On the one hand, they usually recast RTE task to text-to-text prompting formats, which is unnatural and results in a mismatch between the output format at the pre-training time and the inference time for large language models (LLMs). On the other hand, they only utilize surface natural language features and lack consideration of triple semantics in sample selection. These issues are blocking improved performance in ICL for RTE, thus we aim to tackle prompt designing and sample selection challenges simultaneously. To this end, we devise a tabular prompting for RTE (\textsc{TableIE}) which frames RTE task into a table generation task to incorporate explicit structured information into ICL, facilitating conversion of outputs to RTE structures. Then we propose instructive in-context learning (I$^2$CL) which only selects and annotates a few samples considering internal triple semantics in massive unlabeled samples.

Figures (4)

• Figure 1: Illustration of the instance-wise retrieving ICL (a) and our I$^2$CL framework (b). For different test samples (e.g., $t_1$ and $t_2$), the former paradigm retrieves $B$ different samples from the large labeled dataset as corresponding demonstrations $d$, while I$^2$CL only selects and annotates a few samples with annotation budget $B$ for all $M$ test samples.
• Figure 2: Formats of three prompting. The test sample is marked with underline. The outputs of LLMs are highlighted in colors.
• Figure 3: Illustration of the I$^2$CL framework. We aim to measure the similarities of the triple sets contained in two samples and select the most representative samples to annotate based on the whole test data. For instance, sample (b) is very similar to (a) on the surface natural language features while (c) is more similar to (a) in the triple-level semantic features. Moreover, annotate one sample (c) is better than two samples (a) and (b) because (c) contains all the similar triple patterns in (a) and (b).
• Figure 4: Performance of different retrieval models on two benchmarks. We use text-davinci-003 on CoNLL04 and gpt-3.5-turbo-16k on NYT. BERT denotes the original Sentence-BERT without fine-tuning. Gold denotes the fine-tuned model on annotated training data, while Silver denotes the fine-tuned model on training data with pre-extraction results. And B, T, C represent balance, top-k, and coverage strategies, respectively.