Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Dynamic DBSCAN with Euler Tour Sequences

Seiyun Shin, Ilan Shomorony, Peter Macgregor

TL;DR

The paper tackles the bottlenecks of batch DBSCAN in dynamic, evolving datasets by introducing DynamicDBSCAN, which uses locality-sensitive hashing for fast core-point estimation and Euler Tour Trees to maintain a dynamic spanning forest of core points. This combination enables online updates with a poly-logarithmic time bound per insertion or deletion, while preserving near-optimal density-level-set accuracy. The authors provide rigorous time-complexity analyses, correctness proofs, and Hausdorff-distance guarantees for density-set approximation, alongside comprehensive empirical results showing substantial speedups and competitive clustering quality on streaming data. The work enables scalable, real-time density-based clustering for large, evolving data and points toward future extensions to other density-based methods such as HDBSCAN.

Abstract

We propose a fast and dynamic algorithm for Density-Based Spatial Clustering of Applications with Noise (DBSCAN) that efficiently supports online updates. Traditional DBSCAN algorithms, designed for batch processing, become computationally expensive when applied to dynamic datasets, particularly in large-scale applications where data continuously evolves. To address this challenge, our algorithm leverages the Euler Tour Trees data structure, enabling dynamic clustering updates without the need to reprocess the entire dataset. This approach preserves a near-optimal accuracy in density estimation, as achieved by the state-of-the-art static DBSCAN method (Esfandiari et al., 2021) Our method achieves an improved time complexity of $O(d \log^3(n) + \log^4(n))$ for every data point insertion and deletion, where $n$ and $d$ denote the total number of updates and the data dimension, respectively. Empirical studies also demonstrate significant speedups over conventional DBSCANs in real-time clustering of dynamic datasets, while maintaining comparable or superior clustering quality.

Dynamic DBSCAN with Euler Tour Sequences

TL;DR

The paper tackles the bottlenecks of batch DBSCAN in dynamic, evolving datasets by introducing DynamicDBSCAN, which uses locality-sensitive hashing for fast core-point estimation and Euler Tour Trees to maintain a dynamic spanning forest of core points. This combination enables online updates with a poly-logarithmic time bound per insertion or deletion, while preserving near-optimal density-level-set accuracy. The authors provide rigorous time-complexity analyses, correctness proofs, and Hausdorff-distance guarantees for density-set approximation, alongside comprehensive empirical results showing substantial speedups and competitive clustering quality on streaming data. The work enables scalable, real-time density-based clustering for large, evolving data and points toward future extensions to other density-based methods such as HDBSCAN.

Abstract

We propose a fast and dynamic algorithm for Density-Based Spatial Clustering of Applications with Noise (DBSCAN) that efficiently supports online updates. Traditional DBSCAN algorithms, designed for batch processing, become computationally expensive when applied to dynamic datasets, particularly in large-scale applications where data continuously evolves. To address this challenge, our algorithm leverages the Euler Tour Trees data structure, enabling dynamic clustering updates without the need to reprocess the entire dataset. This approach preserves a near-optimal accuracy in density estimation, as achieved by the state-of-the-art static DBSCAN method (Esfandiari et al., 2021) Our method achieves an improved time complexity of for every data point insertion and deletion, where and denote the total number of updates and the data dimension, respectively. Empirical studies also demonstrate significant speedups over conventional DBSCANs in real-time clustering of dynamic datasets, while maintaining comparable or superior clustering quality.

Paper Structure

This paper contains 21 sections, 9 theorems, 18 equations, 2 figures, 2 tables, 2 algorithms.

Table of Contents

  1. INTRODUCTION
  2. PRELIMINARIES
  3. DBSCAN
  4. Dynamic Forests
  5. DYNAMIC DBSCAN ALGORITHM
  6. THEORETICAL RESULTS
  7. Time Complexity
  8. Correctness of Dynamic Updates
  9. Approximation of Density Level Sets
  10. Assumptions and Parameter Setup
  11. Density Level Set Estimation
  12. EMPIRICAL RESULTS
  13. Experimental setup.
  14. Evaluation methods.
  15. Results and analysis.
  16. ...and 6 more sections

Key Result

Lemma 1

Given ${\varepsilon} > 0$, the following holds for any two points ${\bm{x}}, {\bm{y}} \in \mathbb{R}^d$ and for a hash function $h$:

Figures (2)

  • Figure 1: Illustration of a graph constructed by the DynamicDBSCAN algorithm: Red points represent core points, and each shaded region corresponds to a separate hash bucket. Edge colors match the hash bucket they belong to. Within each hash bucket, a path is added on the core points unless adding an edge would introduce a cycle into the graph.
  • Figure 2: Comparison on the blobs dataset. (a) The running time of each algorithm. (b) The ARI for each algorithm when data points are added in a random order. (c) The ARI for each algorithm when data points are added cluster-by-cluster.

Theorems & Definitions (25)

  • Definition 1
  • Definition 2: Density Level Set
  • Definition 3: Hash Function
  • Lemma 1: esfandiari2021almost
  • Definition 4: Core Points
  • Theorem 1
  • proof
  • Remark 1: Comparison to esfandiari2021almost
  • Remark 2: Memory Complexity
  • Theorem 2
  • ...and 15 more