A Comparative Study of Offline Models and Online LLMs in Fake News Detection

TL;DR

This paper tackles real-time fake news detection by comparing traditional offline models with online LLMs. It first demonstrates the limitations of offline approaches when faced with rapid misinformation shifts and real-time data streams, using a continuously updated dataset from 2023–2024. It then evaluates zero-shot online LLMs (ChatGPT-4o, Claude 3.5 Sonnet, Llama 3.1, Gemini 1.5 Pro) on the same real-time data, showing competitive to superior performance and highlighting domain-specific strengths and limitations. The findings advocate for transitioning to online LLMs—possibly in hybrid systems—for scalable, adaptive, and explainable real-time misinformation detection, with public accessibility enabling broad deployment and democratization of tooling.

Abstract

Fake news detection remains a critical challenge in today's rapidly evolving digital landscape, where misinformation can spread faster than ever before. Traditional fake news detection models often rely on static datasets and auxiliary information, such as metadata or social media interactions, which limits their adaptability to real-time scenarios. Recent advancements in Large Language Models (LLMs) have demonstrated significant potential in addressing these challenges due to their extensive pre-trained knowledge and ability to analyze textual content without relying on auxiliary data. However, many of these LLM-based approaches are still rooted in static datasets, with limited exploration into their real-time processing capabilities. This paper presents a systematic evaluation of both traditional offline models and state-of-the-art LLMs for real-time fake news detection. We demonstrate the limitations of existing offline models, including their inability to adapt to dynamic misinformation patterns. Furthermore, we show that newer LLM models with online capabilities, such as GPT-4, Claude, and Gemini, are better suited for detecting emerging fake news in real-time contexts. Our findings emphasize the importance of transitioning from offline to online LLM models for real-time fake news detection. Additionally, the public accessibility of LLMs enhances their scalability and democratizes the tools needed to combat misinformation. By leveraging real-time data, our work marks a significant step toward more adaptive, effective, and scalable fake news detection systems.