TRUST-VL: An Explainable News Assistant for General Multimodal Misinformation Detection

TL;DR

TRUST-VL tackles multimodal misinformation by unifying textual, visual, and cross-modal reasoning within a single, explainable vision-language system. It introduces the QAVA module to generate task-specific visual cues and TRUST-Instruct to provide structured reasoning chains aligned with fact-checking workflows. A three-stage training pipeline and extensive cross-dataset evaluation demonstrate state-of-the-art performance and strong generalization to unseen distortions, with interpretable explanations. This work advances robust, evidence-grounded misinformation detection applicable to real-world settings by bridging perceptual analysis and evidential reasoning.

Abstract

Multimodal misinformation, encompassing textual, visual, and cross-modal distortions, poses an increasing societal threat that is amplified by generative AI. Existing methods typically focus on a single type of distortion and struggle to generalize to unseen scenarios. In this work, we observe that different distortion types share common reasoning capabilities while also requiring task-specific skills. We hypothesize that joint training across distortion types facilitates knowledge sharing and enhances the model's ability to generalize. To this end, we introduce TRUST-VL, a unified and explainable vision-language model for general multimodal misinformation detection. TRUST-VL incorporates a novel Question-Aware Visual Amplifier module, designed to extract task-specific visual features. To support training, we also construct TRUST-Instruct, a large-scale instruction dataset containing 198K samples featuring structured reasoning chains aligned with human fact-checking workflows. Extensive experiments on both in-domain and zero-shot benchmarks demonstrate that TRUST-VL achieves state-of-the-art performance, while also offering strong generalization and interpretability.

Figures (12)

• Figure 1: Examples of different distortion types in multimodal misinformation.
• Figure 2: Overview of shared and specialized reasoning involved across misinformation detection tasks.
• Figure 3: Architecture of TRUST-VL. Given an image-text pair and associated evidence, TRUST-VL first encodes multimodal inputs through vision and text encoders. Other than projecting the visual features into general visual tokens, we also leverage the Question-Aware Visual Amplifier module, which utilizes a set of randomly initialized learnable tokens conditioned on task-oriented questions to generate task-oriented visual tokens. Finally, TRUST-VL outputs a structured and explainable detection judgment.
• Figure 4: Construction of TRUST-Instruct using structured reasoning template. TRUST-Instruct comprises 198K diverse samples spanning various distortion types, each annotated with rich, step-by-step reasoning chains.
• Figure 5: Progressive training strategy.