Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Effective and Unsupervised Social Event Detection and Evolution via RAG and Structural Entropy

Qitong Liu, Hao Peng, Zuchen Li, Xihang Meng, Ziyu Yang, Jiting Li, Li Sun, Philip S. Yu

TL;DR

RagSEDE tackles unsupervised social event detection and evolution in open-world social streams by combining a key message sampling pipeline, a retrieval-augmented generation detector with a growing event knowledge base, and a structural-entropy driven evolution module. The approach reduces noise and computation via KMS, injects global semantic cues through RAG, and dynamically tracks evolving event keywords with inheritance and forgetting mechanisms. Empirical results on Event2012 and Event2018 show RagSEDE outperforms strong baselines in SED and uniquely provides SEE with coherent keyword evolution summaries. The work advances open-world event analysis by unifying scalable sampling, knowledge-base-guided PLMs, and structural entropy theory, with practical implications for timely event monitoring and information retrieval.

Abstract

With the growing scale of social media, social event detection and evolution modeling have attracted increasing attention. Graph neural networks (GNNs) and transformer-based pre-trained language models (PLMs) have become mainstream approaches in this area. However, existing methods still face three major challenges. First, the sheer volume of social media messages makes learning resource-intensive. Second, the fragmentation of social media messages often impedes the model's ability to capture a comprehensive view of the events. Third, the lack of structured temporal context has hindered the development of effective models for event evolution, limiting users' access to event information. To address these challenges, we propose a foundation model for unsupervised Social Event Detection and Evolution, namely RagSEDE. Specifically, RagSEDE introduces a representativeness- and diversity-driven sampling strategy to extract key messages from massive social streams, significantly reducing noise and computational overhead. It further establishes a novel paradigm based on Retrieval Augmented Generation (RAG) that enhances PLMs in detecting events while simultaneously constructing and maintaining an evolving event knowledge base. Finally, RagSEDE leverages structural information theory to dynamically model event evolution keywords for the first time. Extensive experiments on two public datasets demonstrate the superiority of RagSEDE in open-world social event detection and evolution.

Effective and Unsupervised Social Event Detection and Evolution via RAG and Structural Entropy

TL;DR

RagSEDE tackles unsupervised social event detection and evolution in open-world social streams by combining a key message sampling pipeline, a retrieval-augmented generation detector with a growing event knowledge base, and a structural-entropy driven evolution module. The approach reduces noise and computation via KMS, injects global semantic cues through RAG, and dynamically tracks evolving event keywords with inheritance and forgetting mechanisms. Empirical results on Event2012 and Event2018 show RagSEDE outperforms strong baselines in SED and uniquely provides SEE with coherent keyword evolution summaries. The work advances open-world event analysis by unifying scalable sampling, knowledge-base-guided PLMs, and structural entropy theory, with practical implications for timely event monitoring and information retrieval.

Abstract

With the growing scale of social media, social event detection and evolution modeling have attracted increasing attention. Graph neural networks (GNNs) and transformer-based pre-trained language models (PLMs) have become mainstream approaches in this area. However, existing methods still face three major challenges. First, the sheer volume of social media messages makes learning resource-intensive. Second, the fragmentation of social media messages often impedes the model's ability to capture a comprehensive view of the events. Third, the lack of structured temporal context has hindered the development of effective models for event evolution, limiting users' access to event information. To address these challenges, we propose a foundation model for unsupervised Social Event Detection and Evolution, namely RagSEDE. Specifically, RagSEDE introduces a representativeness- and diversity-driven sampling strategy to extract key messages from massive social streams, significantly reducing noise and computational overhead. It further establishes a novel paradigm based on Retrieval Augmented Generation (RAG) that enhances PLMs in detecting events while simultaneously constructing and maintaining an evolving event knowledge base. Finally, RagSEDE leverages structural information theory to dynamically model event evolution keywords for the first time. Extensive experiments on two public datasets demonstrate the superiority of RagSEDE in open-world social event detection and evolution.
Paper Structure (43 sections, 20 equations, 7 figures, 7 tables, 2 algorithms)

This paper contains 43 sections, 20 equations, 7 figures, 7 tables, 2 algorithms.

Table of Contents

  1. Introduction
  2. Preliminaries
  3. SED
  4. Structural Entropy
  5. Methodology
  6. Key Message Sampling (KMS)
  7. Anchor Construction
  8. Key Message Sampling Strategy
  9. Representativeness Score.
  10. Diversity Score.
  11. Combined Score.
  12. RAG-based Event Detection
  13. Evaluation-LLM for Knowledge Base Construction
  14. Detection-LLM for SED
  15. Knowledge Base Maintenance over Time (KBM)
  16. ...and 28 more sections

Figures (7)

  • Figure 1: Illustration comparison of the existing event analysis model and our proposed RagSEDE.
  • Figure 2: The proposed RagSEDE framework.
  • Figure 3: Sensitivity of hyperparameter $\tau$ on four blocks.
  • Figure 4: Case study for SEE of RagSEDE. Evolution keywords of an Event about Cyclone Nilam from the Event2012 dataset.
  • Figure 5: Running time comparison on the largest blocks.
  • ...and 2 more figures

Theorems & Definitions (1)

  • definition 1: MERGE Operator li2016structural