Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

AMPLE: Emotion-Aware Multimodal Fusion Prompt Learning for Fake News Detection

Xiaoman Xu, Xiangrun Li, Taihang Wang, Ye Jiang

TL;DR

The proposed AMPLE framework demonstrates strong performance on two public datasets in both few-shot and data-rich settings, with results indicating the potential of emotional aspects in fake news detection.

Abstract

Detecting fake news in large datasets is challenging due to its diversity and complexity, with traditional approaches often focusing on textual features while underutilizing semantic and emotional elements. Current methods also rely heavily on large annotated datasets, limiting their effectiveness in more nuanced analysis. To address these challenges, this paper introduces Emotion-\textbf{A}ware \textbf{M}ultimodal Fusion \textbf{P}rompt \textbf{L}\textbf{E}arning (\textbf{AMPLE}) framework to address the above issue by combining text sentiment analysis with multimodal data and hybrid prompt templates. This framework extracts emotional elements from texts by leveraging sentiment analysis tools. It then employs Multi-Head Cross-Attention (MCA) mechanisms and similarity-aware fusion methods to integrate multimodal data. The proposed AMPLE framework demonstrates strong performance on two public datasets in both few-shot and data-rich settings, with results indicating the potential of emotional aspects in fake news detection. Furthermore, the study explores the impact of integrating large language models with this method for text sentiment extraction, revealing substantial room for further improvement. The code can be found at :\url{https://github.com/xxm1215/MMM2025_few-shot/

AMPLE: Emotion-Aware Multimodal Fusion Prompt Learning for Fake News Detection

TL;DR

The proposed AMPLE framework demonstrates strong performance on two public datasets in both few-shot and data-rich settings, with results indicating the potential of emotional aspects in fake news detection.

Abstract

Detecting fake news in large datasets is challenging due to its diversity and complexity, with traditional approaches often focusing on textual features while underutilizing semantic and emotional elements. Current methods also rely heavily on large annotated datasets, limiting their effectiveness in more nuanced analysis. To address these challenges, this paper introduces Emotion-\textbf{A}ware \textbf{M}ultimodal Fusion \textbf{P}rompt \textbf{L}\textbf{E}arning (\textbf{AMPLE}) framework to address the above issue by combining text sentiment analysis with multimodal data and hybrid prompt templates. This framework extracts emotional elements from texts by leveraging sentiment analysis tools. It then employs Multi-Head Cross-Attention (MCA) mechanisms and similarity-aware fusion methods to integrate multimodal data. The proposed AMPLE framework demonstrates strong performance on two public datasets in both few-shot and data-rich settings, with results indicating the potential of emotional aspects in fake news detection. Furthermore, the study explores the impact of integrating large language models with this method for text sentiment extraction, revealing substantial room for further improvement. The code can be found at :\url{https://github.com/xxm1215/MMM2025_few-shot/

Paper Structure

This paper contains 21 sections, 7 equations, 4 figures, 5 tables.

Table of Contents

  1. Introduction
  2. Related work
  3. Sentiment-based FND
  4. Prompt Learning FND
  5. Methodology
  6. Feature Representation
  7. Emotional Elements Extraction
  8. Multi-grained Feature Fusion
  9. Prompt Learning
  10. Task Learning
  11. Experiment
  12. Dataset
  13. Implementation Details
  14. Baseline
  15. Results
  16. ...and 6 more sections

Figures (4)

  • Figure 1: This example illustrates the differences in emotion between real and fake news, with various colors representing distinct emotional states. The example is annotated with positive and negative sentiments based on a lexicon using the LIWC tool.
  • Figure 2: The overall architecture of the proposed AMPLE.
  • Figure 3: Experimental results with different $\alpha$ values.
  • Figure 4: Comparison of F1 scores between modal fusion Strategy A and B.