Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Unveiling Implicit Table Knowledge with Question-Then-Pinpoint Reasoner for Insightful Table Summarization

Kwangwook Seo, Jinyoung Yeo, Dongha Lee

TL;DR

This work focuses on building a plug-and-play table reasoner that can self-question the insightful knowledge and answer it by faithfully pinpointing evidence on the table to provide explainable guidance for the summarizer.

Abstract

Implicit knowledge hidden within the explicit table cells, such as data insights, is the key to generating a high-quality table summary. However, unveiling such implicit knowledge is a non-trivial task. Due to the complex nature of structured tables, it is challenging even for large language models (LLMs) to mine the implicit knowledge in an insightful and faithful manner. To address this challenge, we propose a novel table reasoning framework Question-then-Pinpoint. Our work focuses on building a plug-and-play table reasoner that can self-question the insightful knowledge and answer it by faithfully pinpointing evidence on the table to provide explainable guidance for the summarizer. To train a reliable reasoner, we collect table knowledge by guiding a teacher LLM to follow the coarse-to-fine reasoning paths and refine it through two quality enhancement strategies to selectively distill the high-quality knowledge to the reasoner. Extensive experiments on two table summarization datasets, including our newly proposed InsTaSumm, validate the general effectiveness of our framework.

Unveiling Implicit Table Knowledge with Question-Then-Pinpoint Reasoner for Insightful Table Summarization

TL;DR

This work focuses on building a plug-and-play table reasoner that can self-question the insightful knowledge and answer it by faithfully pinpointing evidence on the table to provide explainable guidance for the summarizer.

Abstract

Implicit knowledge hidden within the explicit table cells, such as data insights, is the key to generating a high-quality table summary. However, unveiling such implicit knowledge is a non-trivial task. Due to the complex nature of structured tables, it is challenging even for large language models (LLMs) to mine the implicit knowledge in an insightful and faithful manner. To address this challenge, we propose a novel table reasoning framework Question-then-Pinpoint. Our work focuses on building a plug-and-play table reasoner that can self-question the insightful knowledge and answer it by faithfully pinpointing evidence on the table to provide explainable guidance for the summarizer. To train a reliable reasoner, we collect table knowledge by guiding a teacher LLM to follow the coarse-to-fine reasoning paths and refine it through two quality enhancement strategies to selectively distill the high-quality knowledge to the reasoner. Extensive experiments on two table summarization datasets, including our newly proposed InsTaSumm, validate the general effectiveness of our framework.
Paper Structure (55 sections, 6 equations, 5 figures, 22 tables, 1 algorithm)

This paper contains 55 sections, 6 equations, 5 figures, 22 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Preliminaries
  3. Input Table Serialization
  4. Problem Formulation
  5. Proposed Framework
  6. Coarse-to-Fine Knowledge Mining
  7. Knowledge Quality Enhancement
  8. Factuality Verification
  9. Importance Scoring
  10. QtP Reasoner Training
  11. Aspect-focused Question Generation Task
  12. Evidence-focused Insight Generation Task
  13. Experiments
  14. Datasets
  15. In-domain
  16. ...and 40 more sections

Figures (5)

  • Figure 1: An example of implicit table knowledge that should be unveiled from explicitly stated table cells to generate the target summary.
  • Figure 2: Overview of our framework. We (a) leverage LLM to collect diverse aspects of knowledge from the table and reference summary. For the collected knowledge, we (b) apply two quality enhancement strategy to construct a high-quality dataset for (c) training a reliable table reasoner $\phi$. During the inference phase (bottom right), the output insight $\mathcal{I}$ from the reasoner is provided to the summarizer $\theta$ as an additional input to guide the summarization.
  • Figure 3: Pairwise summary quality comparison results on InsTaSumm using GPT-3.5 as backbone summarizer. We report the win percentage of QtP Reasoner.
  • Figure 4: Summarization results with different $k$
  • Figure 5: Annotator interface of human evaluation on reasoner generated knowledge quality