Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Beyond Extraction: Contextualising Tabular Data for Efficient Summarisation by Language Models

Uday Allu, Biddwan Ahmed, Vishesh Tripathi

TL;DR

The paper tackles the challenge of accurate complex table query handling in Retrieval-Augmented Generation (RAG) for PDFs by introducing a two-fold strategy: storing PDFs in a PDF vector database and separately extracting and context-enriching tables via header-value concatenation. It then uses a fine-tuned Llama-2-chat for summarisation within the RAG pipeline and further augments the enriched data with ChatGPT-3.5 in a one-shot prompt, integrating the results back into the retrieval store. Evaluated on 200 queries (170 table-oriented, 30 text), the approach yields substantial gains over a text-only baseline, with the best performance achieved when enriched text is parsed by ChatGPT-3.5 Turbo (overall 66%). The findings demonstrate that context-aware tabular enrichment combined with advanced language-model augmentation can significantly improve the precision of complex table queries, offering practical benefits for policy-document summarisation and similar information-retrieval tasks.

Abstract

The conventional use of the Retrieval-Augmented Generation (RAG) architecture has proven effective for retrieving information from diverse documents. However, challenges arise in handling complex table queries, especially within PDF documents containing intricate tabular structures.This research introduces an innovative approach to enhance the accuracy of complex table queries in RAG-based systems. Our methodology involves storing PDFs in the retrieval database and extracting tabular content separately. The extracted tables undergo a process of context enrichment, concatenating headers with corresponding values. To ensure a comprehensive understanding of the enriched data, we employ a fine-tuned version of the Llama-2-chat language model for summarisation within the RAG architecture. Furthermore, we augment the tabular data with contextual sense using the ChatGPT 3.5 API through a one-shot prompt. This enriched data is then fed into the retrieval database alongside other PDFs. Our approach aims to significantly improve the precision of complex table queries, offering a promising solution to a longstanding challenge in information retrieval.

Beyond Extraction: Contextualising Tabular Data for Efficient Summarisation by Language Models

TL;DR

The paper tackles the challenge of accurate complex table query handling in Retrieval-Augmented Generation (RAG) for PDFs by introducing a two-fold strategy: storing PDFs in a PDF vector database and separately extracting and context-enriching tables via header-value concatenation. It then uses a fine-tuned Llama-2-chat for summarisation within the RAG pipeline and further augments the enriched data with ChatGPT-3.5 in a one-shot prompt, integrating the results back into the retrieval store. Evaluated on 200 queries (170 table-oriented, 30 text), the approach yields substantial gains over a text-only baseline, with the best performance achieved when enriched text is parsed by ChatGPT-3.5 Turbo (overall 66%). The findings demonstrate that context-aware tabular enrichment combined with advanced language-model augmentation can significantly improve the precision of complex table queries, offering practical benefits for policy-document summarisation and similar information-retrieval tasks.

Abstract

The conventional use of the Retrieval-Augmented Generation (RAG) architecture has proven effective for retrieving information from diverse documents. However, challenges arise in handling complex table queries, especially within PDF documents containing intricate tabular structures.This research introduces an innovative approach to enhance the accuracy of complex table queries in RAG-based systems. Our methodology involves storing PDFs in the retrieval database and extracting tabular content separately. The extracted tables undergo a process of context enrichment, concatenating headers with corresponding values. To ensure a comprehensive understanding of the enriched data, we employ a fine-tuned version of the Llama-2-chat language model for summarisation within the RAG architecture. Furthermore, we augment the tabular data with contextual sense using the ChatGPT 3.5 API through a one-shot prompt. This enriched data is then fed into the retrieval database alongside other PDFs. Our approach aims to significantly improve the precision of complex table queries, offering a promising solution to a longstanding challenge in information retrieval.
Paper Structure (13 sections, 3 figures, 1 table)

This paper contains 13 sections, 3 figures, 1 table.

Table of Contents

  1. Introduction
  2. Methodology
  3. Model Used in RAG Architecture
  4. Dataset
  5. Experiment Setup
  6. Query Processing
  7. Data Preparation
  8. Integration with Chat-GPT
  9. Results
  10. Normal Existing Pipeline
  11. Table Extracting Separately and Context Enrichment
  12. Parsing Extracted Text to Chat-GPT 3.5 API
  13. Conclusions

Figures (3)

  • Figure 1: The figure illustrates the Tabular Data Enhancement Assistant's task, enhancing clarity by adding headings to row values and generating coherent sentences for tabular data extracted from PDFs. It provides a guide for the assistant, highlighting the desired outcome of improving tabular content understandability.
  • Figure 2: Visualizes query distribution in the experimental dataset: 170 complex table queries and 30 simple text queries out of a total of 200. Offers a balanced evaluation of Retrieval-Augmented Generation (RAG) architecture for both textual and tabular dimensions
  • Figure 3: Figure Represents presents the architectural diagram illustrating the experimental setup. The architecture is designed to demonstrate the workflow of our approach, showcasing the key components involved in enhancing the accuracy of complex table queries within the Retrieval-Augmented Generation (RAG) framework