R1-RE: Cross-Domain Relation Extraction with RLVR

TL;DR

The paper tackles the poor cross-domain generalization of relation extraction by reframing RE as a human-like reasoning task guided by annotation guidelines. It introduces R1-RE, a reinforcement learning with verifiable reward (RLVR) framework that leverages group policy optimization (GRPO) to elicit step-by-step annotation-like reasoning from small LLMs. Through experiments on SemEval-2010 and the private MDKG dataset, R1-RE-7B achieves substantial out-of-domain gains and, on MDKG, performance competitive with GPT-4o, while analysis shows emergence of human-like reasoning and robust generalization when incorporating additional data. The work demonstrates that aligning LLM reasoning with human annotation workflows can significantly enhance cross-domain RE and suggests promising directions for scaling to triplet extraction and larger models, as well as broader evaluation on diverse tasks.

Abstract

Relation extraction (RE) is a core task in natural language processing. Traditional approaches typically frame RE as a supervised learning problem, directly mapping context to labels-an approach that often suffers from poor out-of-domain (OOD) generalization. Inspired by the workflow of human annotators, we reframe RE as a reasoning task guided by annotation guidelines and introduce R1-RE, the first reinforcement learning with verifiable reward (RLVR) framework for RE tasks. Our method elicits the reasoning abilities of small language models for annotation tasks, resulting in significantly improved OOD robustness. We evaluate our approach on the public Sem-2010 dataset and a private MDKG dataset. The R1-RE-7B model attains an average OOD accuracy of approximately 70%, on par with leading proprietary models such as GPT-4o. Additionally, our comprehensive analysis provides novel insights into the training dynamics and emergent reasoning behaviors of the RLVR paradigm for RE.

Figures (7)

• Figure 1: Testing accuracy on the MDKG dataset for R1-RE-7B trained on the Sem-2010 dataset, compared with other models. Detailed results are provided in Table \ref{['tab:RCmain']}.
• Figure 2: (a) Test accuracy on two RE datasets with varying numbers of few-shot examples. (b) In-domain and out-of-domain test accuracy of the SFT model using a naive prompt (Figure \ref{['fig:human']}) versus a prompt incorporating the annotation guide (Figure \ref{['box:RC_template']}).
• Figure 3: A comparison of existing RE training paradigm and the annotation process of human annotators.
• Figure 4: The prompt for RC tasks.
• Figure 5: (a) Training dynamics of R1-RE (Sem-2010), with the left y-axis representing the training reward and the right y-axis showing both in-domain and out-of-domain test accuracy. (b) Response length of R1-RE-7B models during training.