Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Distributed Retrieval-Augmented Generation

Chenhao Xu, Longxiang Gao, Yuan Miao, Xi Zheng

TL;DR

Problem: Centralized RAG on edge devices faces data privacy and scalability challenges. Approach: DRAG provides a decentralized, peer-to-peer RAG framework that distributes knowledge and computation, powered by TARW for topic-aware peer discovery. Contributions: a decentralized architecture, the TARW algorithm, and extensive experiments showing DRAG-TARW achieving near-centralized RAG performance with substantially lower communication overhead. Significance: enables privacy-preserving, scalable knowledge-grounded generation for privacy-sensitive edge applications such as healthcare and autonomous systems.

Abstract

As large language models (LLMs) become increasingly adopted on edge devices, Retrieval-Augmented Generation (RAG) is gaining prominence as a solution to address factual deficiencies and hallucinations by integrating external knowledge. However, centralized RAG architectures face significant challenges in data privacy and scalability. For instance, smart healthcare services often rely on collecting sensitive patient data and building a centralized knowledge base to provide better diagnosis and treatment advice, while privacy concerns significantly impede this process. Besides, maintaining a comprehensive and continuously updated knowledge base is costly, particularly in response to regional epidemics and rapidly mutating viruses. To address these challenges, this paper introduces Distributed Retrieval-Augmented Generation (DRAG), a novel framework that improves data privacy by eliminating the need for a centralized knowledge base and restoring data control to owners. DRAG incorporates a Topic-Aware Random Walk (TARW) algorithm that leverages LLMs to extract query topics and facilitate targeted peer discovery within a peer-to-peer network, enabling efficient knowledge retrieval in decentralized environments. Extensive experiments across three diverse datasets and LLMs demonstrate that DRAG with TARW achieves near-centralized RAG performance by using half as many messages as flooding. The code is available at https://github.com/xuchenhao001/DRAG.

Distributed Retrieval-Augmented Generation

TL;DR

Problem: Centralized RAG on edge devices faces data privacy and scalability challenges. Approach: DRAG provides a decentralized, peer-to-peer RAG framework that distributes knowledge and computation, powered by TARW for topic-aware peer discovery. Contributions: a decentralized architecture, the TARW algorithm, and extensive experiments showing DRAG-TARW achieving near-centralized RAG performance with substantially lower communication overhead. Significance: enables privacy-preserving, scalable knowledge-grounded generation for privacy-sensitive edge applications such as healthcare and autonomous systems.

Abstract

As large language models (LLMs) become increasingly adopted on edge devices, Retrieval-Augmented Generation (RAG) is gaining prominence as a solution to address factual deficiencies and hallucinations by integrating external knowledge. However, centralized RAG architectures face significant challenges in data privacy and scalability. For instance, smart healthcare services often rely on collecting sensitive patient data and building a centralized knowledge base to provide better diagnosis and treatment advice, while privacy concerns significantly impede this process. Besides, maintaining a comprehensive and continuously updated knowledge base is costly, particularly in response to regional epidemics and rapidly mutating viruses. To address these challenges, this paper introduces Distributed Retrieval-Augmented Generation (DRAG), a novel framework that improves data privacy by eliminating the need for a centralized knowledge base and restoring data control to owners. DRAG incorporates a Topic-Aware Random Walk (TARW) algorithm that leverages LLMs to extract query topics and facilitate targeted peer discovery within a peer-to-peer network, enabling efficient knowledge retrieval in decentralized environments. Extensive experiments across three diverse datasets and LLMs demonstrate that DRAG with TARW achieves near-centralized RAG performance by using half as many messages as flooding. The code is available at https://github.com/xuchenhao001/DRAG.
Paper Structure (13 sections, 7 figures, 1 table, 1 algorithm)

This paper contains 13 sections, 7 figures, 1 table, 1 algorithm.

Table of Contents

  1. Introduction
  2. Related Work
  3. RAG Scalability
  4. RAG Privacy and Security
  5. Proposed Model
  6. Distributed RAG
  7. Topic-Aware Random Walk
  8. Further Discussion
  9. Experiments
  10. Experimental Setup
  11. Performance Comparison
  12. Sensitivity Analysis
  13. Conclusion

Figures (7)

  • Figure 1: Overview of the Distributed RAG (DRAG) Architecture. DRAG employs a peer-to-peer network where each peer maintains a local knowledge base and utilizes the Topic-Aware Random Walk (TARW) algorithm for distributed knowledge retrieval. The query process involves local query analysis, knowledge discovery, distributed retrieval, local generation, and result caching.
  • Figure 2: Comparative F1 Scores of DRAG and Centralized RAG Variants Under Varying Knowledge Base Completeness. CRAG-0.7S means the centralized knowledge base contains $70\%$ of knowledge snippets, while CRAG-0.7T means it includes $70\%$ of topics.
  • Figure 3: Impact of network size on F1 score (first row) and message overhead (second row) in DRAG.
  • Figure 4: Convergence of average message counts with increasing query number in DRAG-TARW. Each line represents a different network size, with $20$, $40$, $60$, $80$, and $100$ indicating the number of peers in the network.
  • Figure 5: Impact of peer connectivity on performance (a) and communication cost (b) in DRAG-TARW. Peer connectivity is defined as the number of edges attached from a new node to existing nodes ($2$, $4$, $6$, and $8$), consistent with the Barabási-Albert model.
  • ...and 2 more figures