Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

TaCube: Pre-computing Data Cubes for Answering Numerical-Reasoning Questions over Tabular Data

Fan Zhou, Mengkang Hu, Haoyu Dong, Zhoujun Cheng, Shi Han, Dongmei Zhang

TL;DR

TaCube addresses numerical reasoning over tabular data by pre-computing a data-cube of aggregations and arithmetic, then feeding these cube items alongside the original table into decoder-based PLMs. It introduces cube_ext operators to broaden reasoning beyond simple aggregations, and offers rule-based generation coupled with neural ranking to produce compact, question-sensitive cubes. Empirically, TaCube achieves state-of-the-art or near-SOTA results on TAT-QA and WikiTQ, with especially large gains on numerical reasoning tasks such as sum, average, subtraction, and division. The approach is presented as a general, portable pre-computation technique that can enhance various NR frameworks for table QA.

Abstract

Existing auto-regressive pre-trained language models (PLMs) like T5 and BART, have been well applied to table question answering by UNIFIEDSKG and TAPEX, respectively, and demonstrated state-of-the-art results on multiple benchmarks. However, auto-regressive PLMs are challenged by recent emerging numerical reasoning datasets, such as TAT-QA, due to the error-prone implicit calculation. In this paper, we present TaCube, to pre-compute aggregation/arithmetic results for the table in advance, so that they are handy and readily available for PLMs to answer numerical reasoning questions. TaCube systematically and comprehensively covers a collection of computational operations over table segments. By simply concatenating TaCube to the input sequence of PLMs, it shows significant experimental effectiveness. TaCube promotes the F1 score from 49.6% to 66.2% on TAT-QA and achieves new state-of-the-art results on WikiTQ (59.6% denotation accuracy). TaCube's improvements on numerical reasoning cases are even more notable: on TAT-QA, TaCube promotes the exact match accuracy of BART-large by 39.6% on sum, 52.5% on average, 36.6% on substraction, and 22.2% on division. We believe that TaCube is a general and portable pre-computation solution that can be potentially integrated to various numerical reasoning frameworks

TaCube: Pre-computing Data Cubes for Answering Numerical-Reasoning Questions over Tabular Data

TL;DR

TaCube addresses numerical reasoning over tabular data by pre-computing a data-cube of aggregations and arithmetic, then feeding these cube items alongside the original table into decoder-based PLMs. It introduces cube_ext operators to broaden reasoning beyond simple aggregations, and offers rule-based generation coupled with neural ranking to produce compact, question-sensitive cubes. Empirically, TaCube achieves state-of-the-art or near-SOTA results on TAT-QA and WikiTQ, with especially large gains on numerical reasoning tasks such as sum, average, subtraction, and division. The approach is presented as a general, portable pre-computation technique that can enhance various NR frameworks for table QA.

Abstract

Existing auto-regressive pre-trained language models (PLMs) like T5 and BART, have been well applied to table question answering by UNIFIEDSKG and TAPEX, respectively, and demonstrated state-of-the-art results on multiple benchmarks. However, auto-regressive PLMs are challenged by recent emerging numerical reasoning datasets, such as TAT-QA, due to the error-prone implicit calculation. In this paper, we present TaCube, to pre-compute aggregation/arithmetic results for the table in advance, so that they are handy and readily available for PLMs to answer numerical reasoning questions. TaCube systematically and comprehensively covers a collection of computational operations over table segments. By simply concatenating TaCube to the input sequence of PLMs, it shows significant experimental effectiveness. TaCube promotes the F1 score from 49.6% to 66.2% on TAT-QA and achieves new state-of-the-art results on WikiTQ (59.6% denotation accuracy). TaCube's improvements on numerical reasoning cases are even more notable: on TAT-QA, TaCube promotes the exact match accuracy of BART-large by 39.6% on sum, 52.5% on average, 36.6% on substraction, and 22.2% on division. We believe that TaCube is a general and portable pre-computation solution that can be potentially integrated to various numerical reasoning frameworks
Paper Structure (20 sections, 2 equations, 7 figures, 6 tables)

This paper contains 20 sections, 2 equations, 7 figures, 6 tables.

Table of Contents

  1. Introduction
  2. Preliminary
  3. Data Cube As Pre-computation
  4. Datasets
  5. PLMs for Table QA
  6. TaCube
  7. Data Cube Application in Table QA
  8. Cube Generation
  9. Cube Ranking
  10. Coverage Discussion
  11. Experimental Results and Analysis
  12. Experimental Setup
  13. Main Results
  14. Related Work
  15. Conclusion
  16. ...and 5 more sections

Figures (7)

  • Figure 1: Augmenting auto-regressive PLMs with TaCube. By simply concatenating TaCube to the input sequence, TaCube significantly mitigates the calculation challenge in numerical reasoning over tables.
  • Figure 2: Illustrtion of relationship between $\mathtt{cube}$, $\mathtt{cube_{ext}}$, and cube uncovered cases.
  • Figure 3: Coverage on validation data of tat-qa, where K stands for picking top k pre-computed results as input.
  • Figure 4: outside knowledge
  • Figure 5: non-number pattern
  • ...and 2 more figures