CPTuning: Contrastive Prompt Tuning for Generative Relation Extraction

TL;DR

The paper tackles relation extraction under entity pair overlap (EPO), where multiple relations may hold between the same entities. It introduces CPTuning, a contrastive prompt-tuning framework that reformulates RE as Seq2Seq text-infilling with verbalized relation labels and uses Trie-constrained decoding and a prefix-based beam search to generate and select multiple plausible relations. Two key components are proposed: label smoothing via LBLS and a contrastive loss to calibrate the model’s likelihoods between gold and negative relations, all learned with a T5-large backbone. Experiments on TACRED, TACREV, Re-TACRED, and NYT demonstrate state-of-the-art performance for both single- and multi-relations extraction, showing robust handling of EPO and semantic alignment of verbalized relations with gold semantics. The approach offers advances in practical RE systems and supports multi-relations extraction in downstream knowledge-graph construction and QA tasks.

Abstract

Generative relation extraction (RE) commonly involves first reformulating RE as a linguistic modeling problem easily tackled with pre-trained language models (PLM) and then fine-tuning a PLM with supervised cross-entropy loss. Although having achieved promising performance, existing approaches assume only one deterministic relation between each pair of entities without considering real scenarios where multiple relations may be valid, i.e., entity pair overlap, causing their limited applications. To address this problem, we introduce a novel contrastive prompt tuning method for RE, CPTuning, which learns to associate a candidate relation between two in-context entities with a probability mass above or below a threshold, corresponding to whether the relation exists. Beyond learning schema, CPTuning also organizes RE as a verbalized relation generation task and uses Trie-constrained decoding to ensure a model generates valid relations. It adaptively picks out the generated candidate relations with a high estimated likelihood in inference, thereby achieving multi-relation extraction. We conduct extensive experiments on four widely used datasets to validate our method. Results show that T5-large fine-tuned with CPTuning significantly outperforms previous methods, regardless of single or multiple relations extraction.

Figures (5)

• Figure 1: The standard paradigm of relations extraction.
• Figure 2: The syntax structure of (a) our handcrafted templates and (b) the synthetic source and target texts in the transformed RE task. {sentence}: instance text slot, {entX}: entity slots, {relation}: relation words slot, and {typeX}: entity type slots. Sentinel tokens [X], [Y], and [Z] corrupt an original formalized instance to build a source text. [E] indicates the end of a target text.
• Figure 3: The illustration of the Seq2Seq text-infilling task transformed from RE.
• Figure 4: The illustration of Trie, PGC decoding (a), and LBLS (b). The five relations predefined beforehand are "per:city_of_birth", "per:citiy_of_residence", "per:city_of_death", "per:country_of_birth", and "per:country_of_death", respectively. We contrast the probability mass of sampling a candidate relation with and without LBLS in the (b) bottom.
• Figure 5: CPTuning-s1 performance with varying $\mu$ on TACRED (a) and NYT (b).