HRDE: Retrieval-Augmented Large Language Models for Chinese Health Rumor Detection and Explainability

TL;DR

This work addresses the proliferation of health rumors in Chinese online content by introducing HealthRCN, a large-scale dataset containing 1.12 million health-related rumors, and HRDE, a retrieval-augmented large language model framework for health rumor detection with explainability. HRDE integrates dual retrieval (Elasticsearch and Milvus), HyDE-based evidence re-ranking, and supervised fine-tuning of Qwen models to deliver accurate rumor detection and rich, source-backed explanations. Evaluations show HRDE surpasses several baselines, including GPT-4-1106-Preview, achieving an average accuracy of 91.04% and F1 of 91.58%, while providing interpretable analyses that reference retrieved sources. The work contributes a public Chinese health rumor dataset, a scalable retrieval-augmented detection framework, and insights on parameter choices for evidence retrieval, with potential impact on public health information verification and user trust in health information online.

Abstract

As people increasingly prioritize their health, the speed and breadth of health information dissemination on the internet have also grown. At the same time, the presence of false health information (health rumors) intermingled with genuine content poses a significant potential threat to public health. However, current research on Chinese health rumors still lacks a large-scale, public, and open-source dataset of health rumor information, as well as effective and reliable rumor detection methods. This paper addresses this gap by constructing a dataset containing 1.12 million health-related rumors (HealthRCN) through web scraping of common health-related questions and a series of data processing steps. HealthRCN is the largest known dataset of Chinese health information rumors to date. Based on this dataset, we propose retrieval-augmented large language models for Chinese health rumor detection and explainability (HRDE). This model leverages retrieved relevant information to accurately determine whether the input health information is a rumor and provides explanatory responses, effectively aiding users in verifying the authenticity of health information. In evaluation experiments, we compared multiple models and found that HRDE outperformed them all, including GPT-4-1106-Preview, in rumor detection accuracy and answer quality. HRDE achieved an average accuracy of 91.04% and an F1 score of 91.58%.

Figures (11)

• Figure 1: Framework of HRDE. The sections labeled with black numbers correspond to the process of health rumor detection, while the sections labeled with purple numbers correspond to the process of fine-tuning the large model.
• Figure 2: The Construction Process of HealthRCN
• Figure 3: Prompts for Refutation
• Figure 4: Case one: Mobile phone radiation can cause cancer.
• Figure 5: Case Two: Vitamin C can prevent colds.