Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

ReasonTabQA: A Comprehensive Benchmark for Table Question Answering from Real World Industrial Scenarios

Changzai Pan, Jie Zhang, Kaiwen Wei, Chenshuo Pan, Yu Zhao, Jingwang Huang, Jian Yang, Zhenhe Wu, Haoyang Zeng, Xiaoyan Gu, Weichao Sun, Yanbo Zhai, Yujie Mao, Zhuoru Jiang, Jiang Zhong, Shuangyong Song, Yongxiang Li, Zhongjiang He

TL;DR

ReasonTabQA targets the gap between existing TableQA benchmarks and real-world industrial data by delivering a large-scale, bilingual benchmark with 1,932 tables across 30 domains and 5,523<question> triples, including dual-mode reasoning traces. It introduces TabCodeRL, a reinforcement learning framework using table-specific verifiable rewards to guide executable code generation and robust table reasoning. Experiments across 29 baselines and four TableQA benchmarks show TabCodeRL yields substantial gains for open-source models, yet a persistent performance gap remains on ReasonTabQA, underscoring the inherent complexity of industrial-scale reasoning over multi-sheet and large tables. The work presents a practical resource for enterprise data reasoning and a pathway toward more reliable, transparent AI-assisted BI and ERP decision support, with plans to broaden domain and language coverage in future work.

Abstract

Recent advancements in Large Language Models (LLMs) have significantly catalyzed table-based question answering (TableQA). However, existing TableQA benchmarks often overlook the intricacies of industrial scenarios, which are characterized by multi-table structures, nested headers, and massive scales. These environments demand robust table reasoning through deep structured inference, presenting a significant challenge that remains inadequately addressed by current methodologies. To bridge this gap, we present ReasonTabQA, a large-scale bilingual benchmark encompassing 1,932 tables across 30 industry domains such as energy and automotive. ReasonTabQA provides high-quality annotations for both final answers and explicit reasoning chains, supporting both thinking and no-thinking paradigms. Furthermore, we introduce TabCodeRL, a reinforcement learning method that leverages table-aware verifiable rewards to guide the generation of logical reasoning paths. Extensive experiments on ReasonTabQA and 4 TableQA datasets demonstrate that while TabCodeRL yields substantial performance gains on open-source LLMs, the persistent performance gap on ReasonTabQA underscores the inherent complexity of real-world industrial TableQA.

ReasonTabQA: A Comprehensive Benchmark for Table Question Answering from Real World Industrial Scenarios

TL;DR

ReasonTabQA targets the gap between existing TableQA benchmarks and real-world industrial data by delivering a large-scale, bilingual benchmark with 1,932 tables across 30 domains and 5,523<question> triples, including dual-mode reasoning traces. It introduces TabCodeRL, a reinforcement learning framework using table-specific verifiable rewards to guide executable code generation and robust table reasoning. Experiments across 29 baselines and four TableQA benchmarks show TabCodeRL yields substantial gains for open-source models, yet a persistent performance gap remains on ReasonTabQA, underscoring the inherent complexity of industrial-scale reasoning over multi-sheet and large tables. The work presents a practical resource for enterprise data reasoning and a pathway toward more reliable, transparent AI-assisted BI and ERP decision support, with plans to broaden domain and language coverage in future work.

Abstract

Recent advancements in Large Language Models (LLMs) have significantly catalyzed table-based question answering (TableQA). However, existing TableQA benchmarks often overlook the intricacies of industrial scenarios, which are characterized by multi-table structures, nested headers, and massive scales. These environments demand robust table reasoning through deep structured inference, presenting a significant challenge that remains inadequately addressed by current methodologies. To bridge this gap, we present ReasonTabQA, a large-scale bilingual benchmark encompassing 1,932 tables across 30 industry domains such as energy and automotive. ReasonTabQA provides high-quality annotations for both final answers and explicit reasoning chains, supporting both thinking and no-thinking paradigms. Furthermore, we introduce TabCodeRL, a reinforcement learning method that leverages table-aware verifiable rewards to guide the generation of logical reasoning paths. Extensive experiments on ReasonTabQA and 4 TableQA datasets demonstrate that while TabCodeRL yields substantial performance gains on open-source LLMs, the persistent performance gap on ReasonTabQA underscores the inherent complexity of real-world industrial TableQA.
Paper Structure (50 sections, 6 equations, 8 figures, 10 tables)

This paper contains 50 sections, 6 equations, 8 figures, 10 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. TableQA Benchmarks.
  4. LLM-based Table Reasoning.
  5. Construction of ReasonTabQA
  6. Table Acquisition
  7. Table Question Generation
  8. Seed Question and Prompt Construction.
  9. Self-Instruct Generation.
  10. Human Annotation and Filtering.
  11. Table Answer Generation
  12. Dataset Statistics
  13. Structural Complexity.
  14. Domain Distribution.
  15. Response Characteristics.
  16. ...and 35 more sections

Figures (8)

  • Figure 1: The ReasonTabQA dataset consists of industrial level tables, annotated questions, annotated gold-standard answers, and annotated reasoning processes across different reasoning modes (thinking and no-thinking). The generated code is omitted.
  • Figure 2: An overview of the construction pipeline for ReasonTabQA.
  • Figure 3: Distribution of different types of tables in ReasonTabQA. (a) Domain distribution. (b) Proportion of Chinese and English tables. (c) Proportion of complex header tables. (d-e) The row and cell size distribution for all tables. (f) Proportion of sheets number in each directory. (g) Proportion of response length for SFT datasets with different reasoning modes.
  • Figure 4: Overview of TabCodeRL. The TabCodeRL method integrates piecewise discrete rewards with inner-group code semantic similarity rewards to provide granular optimization signals.
  • Figure 5: Case study comparison of reasoning process before and after TabCodeRL.
  • ...and 3 more figures