Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Reasoning by Commented Code for Table Question Answering

Seho Pyo, Jiheon Seok, Jaejin Lee

TL;DR

This work tackles TableQA by addressing the brittleness of linearized table inputs and the lack of verifiable numerical reasoning in LLM-driven methods. It proposes a commented, step-by-step code-generation framework that grounds reasoning in executable Pandas code via explicit planning and operation-level comments, ensuring deterministic computations. Training data are built from WikiTQ with error-guided refinement, and an answer selector fuses code-based execution with an advanced end-to-end model to maximize accuracy, achieving up to $84.3\%$ FM on WikiTQ. The framework improves numerical reliability and interpretability, scales across model sizes, and narrows the gap for smaller open-source models while remaining applicable to real-world, noisy tabular data.

Abstract

Table Question Answering (TableQA) poses a significant challenge for large language models (LLMs) because conventional linearization of tables often disrupts the two-dimensional relationships intrinsic to structured data. Existing methods, which depend on end-to-end answer generation or single-line program queries, typically exhibit limited numerical accuracy and reduced interpretability. This work introduces a commented, step-by-step code-generation framework that incorporates explicit reasoning into the Python program-generation process. The approach decomposes TableQA reasoning into multi-line executable programs with concise natural language comments, thereby promoting clearer reasoning and increasing the likelihood of generating correct code. On the WikiTableQuestions benchmark, the proposed method achieves 70.9\% accuracy using Qwen2.5-Coder-7B-Instruct, surpassing the Repanda baseline (67.6\%). Integrating the proposed framework with a robust end-to-end TableQA model via a lightweight answer-selection mechanism yields further improvements. This combined approach achieves up to 84.3\% accuracy on the WikiTableQuestions benchmark.

Reasoning by Commented Code for Table Question Answering

TL;DR

This work tackles TableQA by addressing the brittleness of linearized table inputs and the lack of verifiable numerical reasoning in LLM-driven methods. It proposes a commented, step-by-step code-generation framework that grounds reasoning in executable Pandas code via explicit planning and operation-level comments, ensuring deterministic computations. Training data are built from WikiTQ with error-guided refinement, and an answer selector fuses code-based execution with an advanced end-to-end model to maximize accuracy, achieving up to FM on WikiTQ. The framework improves numerical reliability and interpretability, scales across model sizes, and narrows the gap for smaller open-source models while remaining applicable to real-world, noisy tabular data.

Abstract

Table Question Answering (TableQA) poses a significant challenge for large language models (LLMs) because conventional linearization of tables often disrupts the two-dimensional relationships intrinsic to structured data. Existing methods, which depend on end-to-end answer generation or single-line program queries, typically exhibit limited numerical accuracy and reduced interpretability. This work introduces a commented, step-by-step code-generation framework that incorporates explicit reasoning into the Python program-generation process. The approach decomposes TableQA reasoning into multi-line executable programs with concise natural language comments, thereby promoting clearer reasoning and increasing the likelihood of generating correct code. On the WikiTableQuestions benchmark, the proposed method achieves 70.9\% accuracy using Qwen2.5-Coder-7B-Instruct, surpassing the Repanda baseline (67.6\%). Integrating the proposed framework with a robust end-to-end TableQA model via a lightweight answer-selection mechanism yields further improvements. This combined approach achieves up to 84.3\% accuracy on the WikiTableQuestions benchmark.
Paper Structure (27 sections, 2 equations, 9 figures, 8 tables)

This paper contains 27 sections, 2 equations, 9 figures, 8 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Text-to-SQL
  4. Code-based Table Question Answering
  5. End-to-end Table Question Answering
  6. Methods
  7. Instruction Design
  8. Constructing Training Data
  9. Answer Selector
  10. Experiments
  11. Datasets
  12. Experimental Setup
  13. Inference and Evaluation Metrics
  14. Results of the Proposed method
  15. Comparison with Prior Methods
  16. ...and 12 more sections

Figures (9)

  • Figure 1: Overview of the proposed framework.
  • Figure 2: Overview of dataset generation.
  • Figure 3: Overview of the answer selector
  • Figure 4: Performance breakdown of the proposed Python-based TableQA reasoning model and end-to-end TableQA reasoning model, Table-R1.
  • Figure 5: Testset 2521
  • ...and 4 more figures