Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Can GRPO Boost Complex Multimodal Table Understanding?

Xiaoqiang Kang, Shengen Wu, Zimu Wang, Yilin Liu, Xiaobo Jin, Kaizhu Huang, Wei Wang, Yutao Yue, Xiaowei Huang, Qiufeng Wang

Abstract

Existing table understanding methods face challenges due to complex table structures and intricate logical reasoning. While supervised finetuning (SFT) dominates existing research, reinforcement learning (RL), such as Group Relative Policy Optimization (GRPO), has shown promise but struggled with low initial policy accuracy and coarse rewards in tabular contexts. In this paper, we introduce Table-R1, a three-stage RL framework that enhances multimodal table understanding through: (1) Warm-up that prompts initial perception and reasoning capabilities, (2) Perception Alignment GRPO (PA-GRPO), which employs continuous Tree-Edit-Distance Similarity (TEDS) rewards for recognizing table structures and contents, and (3) Hint-Completion GRPO (HC-GRPO), which utilizes fine-grained rewards of residual steps based on the hint-guided question. Extensive experiments demonstrate that Table-R1 can boost the model's table reasoning performance obviously on both held-in and held-out datasets, outperforming SFT and GRPO largely. Notably, Qwen2-VL-7B with Table-R1 surpasses larger specific table understanding models (e.g., Table-LLaVA 13B), even achieving comparable performance to the closed-source model GPT-4o on held-in datasets, demonstrating the efficacy of each stage of Table-R1 in overcoming initialization bottlenecks and reward sparsity, thereby advancing robust multimodal table understanding.

Can GRPO Boost Complex Multimodal Table Understanding?

Abstract

Existing table understanding methods face challenges due to complex table structures and intricate logical reasoning. While supervised finetuning (SFT) dominates existing research, reinforcement learning (RL), such as Group Relative Policy Optimization (GRPO), has shown promise but struggled with low initial policy accuracy and coarse rewards in tabular contexts. In this paper, we introduce Table-R1, a three-stage RL framework that enhances multimodal table understanding through: (1) Warm-up that prompts initial perception and reasoning capabilities, (2) Perception Alignment GRPO (PA-GRPO), which employs continuous Tree-Edit-Distance Similarity (TEDS) rewards for recognizing table structures and contents, and (3) Hint-Completion GRPO (HC-GRPO), which utilizes fine-grained rewards of residual steps based on the hint-guided question. Extensive experiments demonstrate that Table-R1 can boost the model's table reasoning performance obviously on both held-in and held-out datasets, outperforming SFT and GRPO largely. Notably, Qwen2-VL-7B with Table-R1 surpasses larger specific table understanding models (e.g., Table-LLaVA 13B), even achieving comparable performance to the closed-source model GPT-4o on held-in datasets, demonstrating the efficacy of each stage of Table-R1 in overcoming initialization bottlenecks and reward sparsity, thereby advancing robust multimodal table understanding.

Paper Structure

This paper contains 34 sections, 8 equations, 6 figures, 10 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Related Work
  3. Multimodal Table Understanding
  4. Reinforcement Learning
  5. Observation
  6. Methodology
  7. Problem Formulation
  8. Table-R1 Framework
  9. Warm-up.
  10. Perception-Alignment GRPO (PA-GRPO).
  11. Hint-Completion GRPO (HC-GRPO).
  12. Unified GRPO-Based Training Objective.
  13. Experiments
  14. Baselines
  15. Datasets and Evaluation
  16. ...and 19 more sections

Figures (6)

  • Figure 1: Comparative analysis of different initial policy accuracy in a group.
  • Figure 2: Overall framework of Table-R1. (1) Warm-up establishes foundational capabilities in both visual perception and reasoning. (2) PA-GRPO refines the model's structural understanding by employing TEDS as a continuous reward. (3) HC-GRPO utilizes fine-grained rewards of residual steps based on the hint-guided question.
  • Figure 3: Examples from the datasets used in PA-GRPO. The highlighted red segments indicate the incorrect predictions. TEDS assigns a continuous score to each output, reflecting the similarity to the golden answer.
  • Figure 4: accuracy reward of PA-GRPO with and without warm-up.
  • Figure 5: Instruction variants for constructing the table recognition task.
  • ...and 1 more figures