Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

PDF Retrieval Augmented Question Answering

Thi Thu Uyen Hoang, Meenakshi Rajendran, Kun Zhang, Yuhan Wu, Viet Anh Nguyen

Abstract

This paper presents an advancement in Question-Answering (QA) systems using a Retrieval Augmented Generation (RAG) framework to enhance information extraction from PDF files. Recognizing the richness and diversity of data within PDFs--including text, images, vector diagrams, graphs, and tables--poses unique challenges for existing QA systems primarily designed for textual content. We seek to develop a comprehensive RAG-based QA system that will effectively address complex multimodal questions, where several data types are combined in the query. This is mainly achieved by refining approaches to processing and integrating non-textual elements in PDFs into the RAG framework to derive precise and relevant answers, as well as fine-tuning large language models to better adapt to our system. We provide an in-depth experimental evaluation of our solution, demonstrating its capability to extract accurate information that can be applied to different types of content across PDFs. This work not only pushes the boundaries of retrieval-augmented QA systems but also lays a foundation for further research in multimodal data integration and processing.

PDF Retrieval Augmented Question Answering

Abstract

This paper presents an advancement in Question-Answering (QA) systems using a Retrieval Augmented Generation (RAG) framework to enhance information extraction from PDF files. Recognizing the richness and diversity of data within PDFs--including text, images, vector diagrams, graphs, and tables--poses unique challenges for existing QA systems primarily designed for textual content. We seek to develop a comprehensive RAG-based QA system that will effectively address complex multimodal questions, where several data types are combined in the query. This is mainly achieved by refining approaches to processing and integrating non-textual elements in PDFs into the RAG framework to derive precise and relevant answers, as well as fine-tuning large language models to better adapt to our system. We provide an in-depth experimental evaluation of our solution, demonstrating its capability to extract accurate information that can be applied to different types of content across PDFs. This work not only pushes the boundaries of retrieval-augmented QA systems but also lays a foundation for further research in multimodal data integration and processing.

Paper Structure

This paper contains 25 sections, 7 equations, 4 figures, 5 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Related Work
  3. Retrieval-Augmented Generation (RAG)
  4. Question Answering (QA) with Language Models (LLMs)
  5. Multimodal Question Answering Systems
  6. Integration of RAG with PDF Processing for QA
  7. Methodology
  8. Objective
  9. System Design
  10. PDF Preprocessing
  11. Embedding and storage
  12. Question Answering
  13. Experiments
  14. Data Collection
  15. Metrics
  16. ...and 10 more sections

Figures (4)

  • Figure 1: PIER-QA: PDF Integrated Enhanced Retrieval Question Answering
  • Figure 2: PDF Preprocessing
  • Figure 3: Question generation prompt
  • Figure 4: Answer generation prompt