Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

MORE-R1: Guiding LVLM for Multimodal Object-Entity Relation Extraction via Stepwise Reasoning with Reinforcement Learning

Xiang Yuan, Xu Chu, Xinrong Chen, Haochen Li, Zonghong Dai, Hongcheng Fan, Xiaoyue Yuan, Weiping Li, Tong Mo

Abstract

Multimodal Object-Entity Relation Extraction (MORE) is a challenging task in information extraction research. It aims to identify relations between visual objects and textual entities, requiring complex multimodal understanding and cross-modal reasoning abilities. Existing methods, mainly classification-based or generation-based without reasoning, struggle to handle complex extraction scenarios in the MORE task and suffer from limited scalability and intermediate reasoning transparency. To address these challenges, we propose MORE-R1, a novel model that introduces explicit stepwise reasoning with Reinforcement Learning (RL) to enable Large Vision-Language Model (LVLM) to address the MORE task effectively. MORE-R1 integrates a two-stage training process, including an initial cold-start training stage with Supervised Fine-Tuning (SFT) and a subsequent RL stage for reasoning ability optimization. In the initial stage, we design an efficient way to automatically construct a high-quality SFT dataset containing fine-grained stepwise reasoning tailored to the MORE task, enabling the model to learn an effective reasoning paradigm. In the subsequent stage, we employ the Group Relative Policy Optimization (GRPO) RL algorithm with a Progressive Sample-Mixing Strategy to stabilize training and further enhance model's reasoning ability on hard samples. Comprehensive experiments on the MORE benchmark demonstrate that MORE-R1 achieves state-of-the-art performance with significant improvement over baselines.

MORE-R1: Guiding LVLM for Multimodal Object-Entity Relation Extraction via Stepwise Reasoning with Reinforcement Learning

Abstract

Multimodal Object-Entity Relation Extraction (MORE) is a challenging task in information extraction research. It aims to identify relations between visual objects and textual entities, requiring complex multimodal understanding and cross-modal reasoning abilities. Existing methods, mainly classification-based or generation-based without reasoning, struggle to handle complex extraction scenarios in the MORE task and suffer from limited scalability and intermediate reasoning transparency. To address these challenges, we propose MORE-R1, a novel model that introduces explicit stepwise reasoning with Reinforcement Learning (RL) to enable Large Vision-Language Model (LVLM) to address the MORE task effectively. MORE-R1 integrates a two-stage training process, including an initial cold-start training stage with Supervised Fine-Tuning (SFT) and a subsequent RL stage for reasoning ability optimization. In the initial stage, we design an efficient way to automatically construct a high-quality SFT dataset containing fine-grained stepwise reasoning tailored to the MORE task, enabling the model to learn an effective reasoning paradigm. In the subsequent stage, we employ the Group Relative Policy Optimization (GRPO) RL algorithm with a Progressive Sample-Mixing Strategy to stabilize training and further enhance model's reasoning ability on hard samples. Comprehensive experiments on the MORE benchmark demonstrate that MORE-R1 achieves state-of-the-art performance with significant improvement over baselines.
Paper Structure (16 sections, 6 equations, 4 figures, 1 table, 1 algorithm)

This paper contains 16 sections, 6 equations, 4 figures, 1 table, 1 algorithm.

Table of Contents

  1. Introduction
  2. Related Works
  3. Multimodal Relation Extraction
  4. Reinforcement Learning for LLMs and LVLMs
  5. Methods
  6. Task Formulation and Overall Framework
  7. Cold-Start Training Stage
  8. Reinforcement Learning Stage
  9. Group Relative Policy Optimization
  10. Progressive Sample-Mixing Strategy
  11. Experiments
  12. Experimental Settings
  13. Main Results
  14. Ablation Studies
  15. Case Studies
  16. ...and 1 more sections

Figures (4)

  • Figure 1: Left side: An example of the MORE task: Identify the relation type between the object and entity from the candidate set. Right side: Different frameworks for the MORE task. Our model belongs to the Generation-based (with reasoning) Method.
  • Figure 2: Overall framework of MORE-R1, which adopts a two-stage training framework. In Stage 1, the cold-start training enables the LVLM to learn the fundamental reasoning paradigm tailored for the MORE task. In Stage 2, RL further enhances the reasoning capability of the LVLM trained by Stage 1. Models marked with a flame icon indicate that they are involved in training, while the snowflake icon denotes parameter-frozen.
  • Figure 3: Results of different sample-mixing strategies on four evaluation metrics. "MORE-R1(Stage 1)" is the baseline that only conducts Stage 1 training. "MORE-R1" reports the result after Stage 2 training with different strategies.
  • Figure 4: Example of actual outputs for Qwen2.5-VL-SFT (purple box), MORE-R1(Stage 1) (green box), and MORE-R1 (blue box). Only MORE-R1 provides the correct answer.