Zero-Indexing Internet Search Augmented Generation for Large Language Models

TL;DR

The paper tackles the problem of keeping large language models up to date by moving from static, pre-indexed RAG to a zero-indexing Internet search augmented generation framework that uses standard search APIs to fetch current information. It introduces a three-part pipeline—parser-LLM for intent detection and keyword extraction, a mixed ranking module to re-order search results and mitigate API biases, and an extractor-LLM for efficient content extraction from fresh HTML content—along with training procedures (SFT and DPO) and a production deployment in 01AI's inference API. Key contributions include the design principles and implementation details of the core components, extensive end-to-end and extractor evaluations, and demonstration of production viability with improved content quality and reduced token costs. The work advances practical, real-time, web-grounded generation and offers a scalable approach for deploying up-to-date information in generative systems with strong robustness to noise and bias.

Abstract

Retrieval augmented generation has emerged as an effective method to enhance large language model performance. This approach typically relies on an internal retrieval module that uses various indexing mechanisms to manage a static pre-processed corpus. However, such a paradigm often falls short when it is necessary to integrate the most up-to-date information that has not been updated into the corpus during generative inference time. In this paper, we explore an alternative approach that leverages standard search engine APIs to dynamically integrate the latest online information (without maintaining any index for any fixed corpus), thereby improving the quality of generated content. We design a collaborative LLM-based paradigm, where we include: (i) a parser-LLM that determines if the Internet augmented generation is demanded and extracts the search keywords if so with a single inference; (ii) a mixed ranking strategy that re-ranks the retrieved HTML files to eliminate bias introduced from the search engine API; and (iii) an extractor-LLM that can accurately and efficiently extract relevant information from the fresh content in each HTML file. We conduct extensive empirical studies to evaluate the performance of this Internet search augmented generation paradigm. The experimental results demonstrate that our method generates content with significantly improved quality. Our system has been successfully deployed in a production environment to serve 01.AI's generative inference requests.

Figures (3)

• Figure 1: A comparison between the standard RAG paradigm and our Internet search augmented generation paradigm.
• Figure 2: An illustration about pre-processing the retrieved HTML file and Extractor LLM processing procedure.
• Figure 3: An illustration Of DPO Training Preparation.