Revolutionizing Retrieval-Augmented Generation with Enhanced PDF Structure Recognition

TL;DR

The paper investigates how PDF parsing quality affects retrieval-augmented generation for professional knowledge QA, comparing a rule-based PyPDF approach with a deep-learning-based ChatDOC PDF Parser. It demonstrates that structure-aware parsing yields more coherent chunks and better retrieval, leading to superior answer quality across extractive and comprehensive questions. Through a sizable real-world dataset and case studies, ChatDOC shows substantial improvements over the baseline, suggesting that advanced PDF structure recognition can substantially revolutionize RAG-based professional QA. These findings underscore the practical impact of robust PDF parsing on downstream LLM reasoning and information retrieval systems.

Abstract

With the rapid development of Large Language Models (LLMs), Retrieval-Augmented Generation (RAG) has become a predominant method in the field of professional knowledge-based question answering. Presently, major foundation model companies have opened up Embedding and Chat API interfaces, and frameworks like LangChain have already integrated the RAG process. It appears that the key models and steps in RAG have been resolved, leading to the question: are professional knowledge QA systems now approaching perfection? This article discovers that current primary methods depend on the premise of accessing high-quality text corpora. However, since professional documents are mainly stored in PDFs, the low accuracy of PDF parsing significantly impacts the effectiveness of professional knowledge-based QA. We conducted an empirical RAG experiment across hundreds of questions from the corresponding real-world professional documents. The results show that, ChatDOC, a RAG system equipped with a panoptic and pinpoint PDF parser, retrieves more accurate and complete segments, and thus better answers. Empirical experiments show that ChatDOC is superior to baseline on nearly 47% of questions, ties for 38% of cases, and falls short on only 15% of cases. It shows that we may revolutionize RAG with enhanced PDF structure recognition.

Figures (16)

• Figure Figure 1: The workflow of Retrieval-Augmented Generation (RAG).
• Figure Figure 2: The process of converting PDFs into retrievable contents.
• Figure Figure 3: Two types of documents in the view of computers.
• Figure Figure 4: Parsing and chunking results of PyPDF on \ref{['par:case1']} (original document: alibaba). Zoom in to see the details.
• Figure Figure 5: An example illustrating the results of the ChatDOC PDF Parser. Zoom in to see the details.