Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

MES-RAG: Bringing Multi-modal, Entity-Storage, and Secure Enhancements to RAG

Pingyu Wu, Daiheng Gao, Jing Tang, Huimin Chen, Wenbo Zhou, Weiming Zhang, Nenghai Yu

TL;DR

Experimental results demonstrate that MES-RAG significantly improves both accuracy and recall, highlighting its effectiveness in advancing the security and utility of question-answering, increasing accuracy to 0.83 (+0.25) on targeted task.

Abstract

Retrieval-Augmented Generation (RAG) improves Large Language Models (LLMs) by using external knowledge, but it struggles with precise entity information retrieval. In this paper, we proposed MES-RAG framework, which enhances entity-specific query handling and provides accurate, secure, and consistent responses. MES-RAG introduces proactive security measures that ensure system integrity by applying protections prior to data access. Additionally, the system supports real-time multi-modal outputs, including text, images, audio, and video, seamlessly integrating into existing RAG architectures. Experimental results demonstrate that MES-RAG significantly improves both accuracy and recall, highlighting its effectiveness in advancing the security and utility of question-answering, increasing accuracy to 0.83 (+0.25) on targeted task. Our code and data are available at https://github.com/wpydcr/MES-RAG.

MES-RAG: Bringing Multi-modal, Entity-Storage, and Secure Enhancements to RAG

TL;DR

Experimental results demonstrate that MES-RAG significantly improves both accuracy and recall, highlighting its effectiveness in advancing the security and utility of question-answering, increasing accuracy to 0.83 (+0.25) on targeted task.

Abstract

Retrieval-Augmented Generation (RAG) improves Large Language Models (LLMs) by using external knowledge, but it struggles with precise entity information retrieval. In this paper, we proposed MES-RAG framework, which enhances entity-specific query handling and provides accurate, secure, and consistent responses. MES-RAG introduces proactive security measures that ensure system integrity by applying protections prior to data access. Additionally, the system supports real-time multi-modal outputs, including text, images, audio, and video, seamlessly integrating into existing RAG architectures. Experimental results demonstrate that MES-RAG significantly improves both accuracy and recall, highlighting its effectiveness in advancing the security and utility of question-answering, increasing accuracy to 0.83 (+0.25) on targeted task. Our code and data are available at https://github.com/wpydcr/MES-RAG.

Paper Structure

This paper contains 21 sections, 12 equations, 2 figures, 7 tables.

Table of Contents

  1. Introduction
  2. Related Works
  3. Framework
  4. Task Definition
  5. Overview
  6. Entity-centric Data Construction
  7. Query Parser
  8. Entities Retrieval
  9. Answer Generation
  10. Experiments
  11. Datasets
  12. Experimental Setup
  13. Baselines
  14. Evaluation Metrics
  15. Implementation Details
  16. ...and 6 more sections

Figures (2)

  • Figure 1: Example workflow of the MES-RAG framework. Left: Entity-centric conversational Q&A and malicious query identification process. Right: Comparison of multi-modal data retrieval and generation.
  • Figure 2: Overview of our framework.