Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Towards Scalable Topic Detection on Web via Simulating Levy Walks Nature of Topics in Similarity Space

Junbiao Pang, Qingming Huang

TL;DR

This work tackles scalable topic detection on the web under a sea of noisy UGC by revealing heavy-tailed similarity patterns within topics that resemble Lévy walks. It introduces a model-free Explore-Exploit framework (EE) to simulate Lévy walks in a $k$-NN$^2$ similarity graph, coupled with Poisson deconvolution (PD) to rank topics without heavy optimization. The method selects seed pages via a Stationary Distribution/ SER mechanism, expands topics through an EE-driven growth rule, and supports multi-granularity topic generation with near-linear complexity. Empirical results on public datasets show LWTG achieves competitive effectiveness with substantially better efficiency and scalability than state-of-the-art baselines, making it suitable for large-scale web monitoring and trend analysis.

Abstract

Organizing a few webpages from social media websites into popular topics is one of the key steps to understand trends on web. Discovering popular topics from web faces a sea of noise webpages which never evolve into popular topics. In this paper, we discover that the similarity values between webpages in a popular topic contain the statistically similar features observed in Levy walks. Consequently, we present a simple, novel, yet very powerful Explore-Exploit (EE) approach to group topics by simulating Levy walks nature in the similarity space. The proposed EE-based topic clustering is an effective and effcient method which is a solid move towards handling a sea of noise webpages. Experiments on two public data sets demonstrate that our approach is not only comparable to the state-of-the-art methods in terms of effectiveness but also significantly outperforms the state-of-the-art methods in terms of efficiency.

Towards Scalable Topic Detection on Web via Simulating Levy Walks Nature of Topics in Similarity Space

TL;DR

This work tackles scalable topic detection on the web under a sea of noisy UGC by revealing heavy-tailed similarity patterns within topics that resemble Lévy walks. It introduces a model-free Explore-Exploit framework (EE) to simulate Lévy walks in a -NN similarity graph, coupled with Poisson deconvolution (PD) to rank topics without heavy optimization. The method selects seed pages via a Stationary Distribution/ SER mechanism, expands topics through an EE-driven growth rule, and supports multi-granularity topic generation with near-linear complexity. Empirical results on public datasets show LWTG achieves competitive effectiveness with substantially better efficiency and scalability than state-of-the-art baselines, making it suitable for large-scale web monitoring and trend analysis.

Abstract

Organizing a few webpages from social media websites into popular topics is one of the key steps to understand trends on web. Discovering popular topics from web faces a sea of noise webpages which never evolve into popular topics. In this paper, we discover that the similarity values between webpages in a popular topic contain the statistically similar features observed in Levy walks. Consequently, we present a simple, novel, yet very powerful Explore-Exploit (EE) approach to group topics by simulating Levy walks nature in the similarity space. The proposed EE-based topic clustering is an effective and effcient method which is a solid move towards handling a sea of noise webpages. Experiments on two public data sets demonstrate that our approach is not only comparable to the state-of-the-art methods in terms of effectiveness but also significantly outperforms the state-of-the-art methods in terms of efficiency.
Paper Structure (20 sections, 7 equations, 10 figures, 4 tables, 3 algorithms)

This paper contains 20 sections, 7 equations, 10 figures, 4 tables, 3 algorithms.

Table of Contents

  1. INTRODUCTION
  2. Related Work
  3. Clustering in Sea of Noises
  4. Hot Topic Detection on Web
  5. Simulating Lévy-Walk Nature of Web Topics in Similarity Space
  6. Projecting Webpages into the Similarity Space
  7. Lévy Walks Nature of Web Topics in the Similarity Space
  8. Generating Topics by Simulating Lévy Walks
  9. Selecting Important Webpages as Seeds
  10. Simulating Lévy Walks by an Exploit-Explore Approach
  11. Generating Lévy Walks-based Topics across Multi-granularity Seeds
  12. Topic Detection on Web by Poisson Deconvolution
  13. Experiment
  14. Evaluation of LWTG
  15. Effectiveness of the EE selection
  16. ...and 5 more sections

Figures (10)

  • Figure 1: The 1-th, 2-th, 3-th and 4-th topics are best fitted by exponentiated Weibull, power law, exponentiated Weibull, power law, exponentiated Weibull, respectively.
  • Figure 2: The distributions of the sorted edges which include both the intra-edges and the inter-edges.
  • Figure 3: The similarity space of a topic consists of the affinity values from both the intra-edges and inter-edges.
  • Figure 4: The role of SER of indicating compactness and homogeneousness seeds.
  • Figure 5: Comparison between HAC and LWT on a toy data. (b) Dendrogram generated by HAC and the corresponding clusters. (c) LWTG generates topics by the EE approach.
  • ...and 5 more figures

Theorems & Definitions (1)

  • Definition 1: Similarity Space of a Topic on Web