A New Hybrid Intelligent Approach for Multimodal Detection of Suspected Disinformation on TikTok

TL;DR

This paper tackles the challenge of detecting disinformation in TikTok videos by proposing a hybrid intelligence framework that couples deep learning-based multimodal feature extraction with fuzzy-logic interpretability. It introduces a Granular Linguistic Model of Phenomena (GLMP) built on Hierarchical Perception Mapping (HPM) to represent psychosociological traits across text, audio, and video, culminating in a final disinformation suspicion score via Choquet Integral aggregation. The methodology encompasses a Conceptualisation Framework, a Multimodal Feature Analyser, a Multimodal Disinformation Detector, and a Prompt Generator to produce detailed, explainable reports. Empirical validation includes context-specific and widespread-context experiments, demonstrating robust discrimination of disinformation signals and yielding high-quality, traceable reports, with limitations and future directions outlined. The work advances explainable, multimodal disinformation detection on social platforms and provides datasets and prompts for broader application and platform expansion.

Abstract

In the context of the rapid dissemination of multimedia content, identifying disinformation on social media platforms such as TikTok represents a significant challenge. This study introduces a hybrid framework that combines the computational power of deep learning with the interpretability of fuzzy logic to detect suspected disinformation in TikTok videos. The methodology is comprised of two core components: a multimodal feature analyser that extracts and evaluates data from text, audio, and video; and a multimodal disinformation detector based on fuzzy logic. These systems operate in conjunction to evaluate the suspicion of spreading disinformation, drawing on human behavioural cues such as body language, speech patterns, and text coherence. Two experiments were conducted: one focusing on context-specific disinformation and the other on the scalability of the model across broader topics. For each video evaluated, high-quality, comprehensive, well-structured reports are generated, providing a detailed view of the disinformation behaviours.

Figures (6)

• Figure 1: Example of a simple GLMP.
• Figure 2: Overview of the proposed approach.
• Figure 3: Dimensions, attributes and measures to detect suspected disinformation.
• Figure 4: Fuzzy sets of disinformer behaviours in the political domain.
• Figure 5: Fuzzy sets of suspicion of disinformation.