RAEmoLLM: Retrieval Augmented LLMs for Cross-Domain Misinformation Detection Using In-Context Learning Based on Emotional Information
Zhiwei Liu, Kailai Yang, Qianqian Xie, Christine de Kock, Sophia Ananiadou, Eduard Hovy
TL;DR
This paper tackles cross-domain misinformation detection by addressing the limitations of domain-specific fine-tuning. It introduces RAEmoLLM, a retrieval-augmented LLM framework that leverages affective information through in-context learning to perform misinformation verification across domains without fine-tuning. The approach comprises an index construction stage using EmoLLaMA-derived affective embeddings, a retrieval stage selecting top-$K$ affective demonstrations from source domains, and an inference stage that prompts LLMs with these demonstrations, sometimes augmented with explicit affective signals. Experimental results on AMTCele, PHEME, and COCO show significant improvements over zero-shot and several few-shot baselines, though performance is dataset-dependent and larger-scale fine-tuning can still surpass it in some cases; the work highlights the potential of emotion-informed retrieval to enhance cross-domain misinformation detection in a resource-efficient manner.
Abstract
Misinformation is prevalent in various fields such as education, politics, health, etc., causing significant harm to society. However, current methods for cross-domain misinformation detection rely on effort- and resource-intensive fine-tuning and complex model structures. With the outstanding performance of LLMs, many studies have employed them for misinformation detection. Unfortunately, they focus on in-domain tasks and do not incorporate significant sentiment and emotion features (which we jointly call {\em affect}). In this paper, we propose RAEmoLLM, the first retrieval augmented (RAG) LLMs framework to address cross-domain misinformation detection using in-context learning based on affective information. RAEmoLLM includes three modules. (1) In the index construction module, we apply an emotional LLM to obtain affective embeddings from all domains to construct a retrieval database. (2) The retrieval module uses the database to recommend top K examples (text-label pairs) from source domain data for target domain contents. (3) These examples are adopted as few-shot demonstrations for the inference module to process the target domain content. The RAEmoLLM can effectively enhance the general performance of LLMs in cross-domain misinformation detection tasks through affect-based retrieval, without fine-tuning. We evaluate our framework on three misinformation benchmarks. Results show that RAEmoLLM achieves significant improvements compared to the other few-shot methods on three datasets, with the highest increases of 15.64%, 31.18%, and 15.73% respectively. This project is available at https://github.com/lzw108/RAEmoLLM.