GAME-ON: Graph Attention Network based Multimodal Fusion for Fake News Detection

TL;DR

GAME-ON introduces a Graph Attention Network–based multimodal fusion framework for fake news detection, addressing inter-modal heterogeneity and overfitting through early fusion in a shared multimodal space and a fully connected multimodal graph. By weighing intra- and inter-modal relationships with a single Graph Attention Layer, it achieves strong Twitter performance and competitive Weibo results while using substantially fewer parameters than baselines. The approach relies on visually grounded object features and BERT-based textual embeddings, concatenated in a two-stage encoder and fused graph to support end-to-end training. Empirical results demonstrate clear advantages in accuracy and model efficiency, highlighting potential applicability to other multimodal classification tasks and deployment in memory-constrained environments.

Abstract

Social media in present times has a significant and growing influence. Fake news being spread on these platforms have a disruptive and damaging impact on our lives. Furthermore, as multimedia content improves the visibility of posts more than text data, it has been observed that often multimedia is being used for creating fake content. A plethora of previous multimodal-based work has tried to address the problem of modeling heterogeneous modalities in identifying fake content. However, these works have the following limitations: (1) inefficient encoding of inter-modal relations by utilizing a simple concatenation operator on the modalities at a later stage in a model, which might result in information loss; (2) training very deep neural networks with a disproportionate number of parameters on small but complex real-life multimodal datasets result in higher chances of overfitting. To address these limitations, we propose GAME-ON, a Graph Neural Network based end-to-end trainable framework that allows granular interactions within and across different modalities to learn more robust data representations for multimodal fake news detection. We use two publicly available fake news datasets, Twitter and Weibo, for evaluations. Our model outperforms on Twitter by an average of 11% and keeps competitive performance on Weibo, within a 2.6% margin, while using 65% fewer parameters than the best comparable state-of-the-art baseline.

Figures (2)

• Figure 1: Overview of the graph construction pipeline for the GAME-ON framework. (a) Given a multimodal post (news sample), taken from the Twitter dataset, we extract individual fully-connected graphs for both the modalities. (b) We find objects from the image and extract their feature representations ${v_i}$. (c) For the textual graph, we first tokenize the text and extract their feature representations ${t_i}$.
• Figure 2: Overview of the GAME-ON framework. It consists of four stages. (a) Each modality is transformed into a unimodal visual and textual graph (subsection \ref{['subsec:feature-encoders']}). (b)To establish common space representations of each modality, Both graphs are routed into fully-connected layers and inter-modality connections are introduced (subsection \ref{['subsec:multimodal-space-graph']}). (c)The multimodal Graph is given to the graph attention layer (L=1) to learn resilient representations (subsection \ref{['subsec:graph-layers']}). (d) The pooling and classification layer are utilized to determine whether or not a news sample is fake (subsection \ref{['subsec:fake-news-detector']}).