Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Distributed-Memory Parallel Algorithms for Fixed-Radius Near Neighbor Graph Construction

Gabriel Raulet, Dmitriy Morozov, Aydin Buluc, Katherine Yelick

TL;DR

The paper tackles the problem of constructing exact fixed-radius near-neighbor graphs in general metric spaces on distributed-memory systems. It introduces batched cover-tree algorithms and two scalable strategies—point-partitioning (systolic) and spatial-partitioning (landmarking)—with a hybrid approach to maximize performance. Key contributions include efficient batched cover-tree construction and querying, ghost-point mechanisms (collective and ring) to capture cross-cell edges, and comprehensive experiments demonstrating large speedups and scalability across diverse datasets and metrics. The work enables exact, scalable near-neighbor graphs for downstream data analysis tasks on datasets with billions of edges, including non-Euclidean distances, which broadens applicability beyond Euclidean-focused methods.

Abstract

Computing fixed-radius near-neighbor graphs is an important first step for many data analysis algorithms. Near-neighbor graphs connect points that are close under some metric, endowing point clouds with a combinatorial structure. As computing power and data acquisition methods advance, diverse sources of large scientific datasets would greatly benefit from scalable solutions to this common subroutine for downstream analysis. Prior work on parallel nearest neighbors has made great progress in problems like k-nearest and approximate nearest neighbor search problems, with particular attention on Euclidean spaces. Yet many applications need exact solutions and non-Euclidean metrics. This paper presents a scalable sparsity-aware distributed memory algorithm using cover trees to compute near-neighbor graphs in general metric spaces. We provide a shared-memory algorithm for cover tree construction and demonstrate its competitiveness with state-of-the-art fixed-radius search data structures. We then introduce two distributed-memory algorithms for the near-neighbor graph problem, a simple point-partitioning strategy and a spatial-partitioning strategy, which leverage the cover tree algorithm on each node. Our algorithms exhibit parallel scaling across a variety of real and synthetic datasets for both traditional and non-traditional metrics. On real world high dimensional datasets with one million points, we achieve speedups up to 678.34x over the state-of-the-art using 1024 cores for graphs with 70 neighbors per vertex (on average), and up to 1590.99x using 4096 cores for graphs with 500 neighbors per vertex (on average).

Distributed-Memory Parallel Algorithms for Fixed-Radius Near Neighbor Graph Construction

TL;DR

The paper tackles the problem of constructing exact fixed-radius near-neighbor graphs in general metric spaces on distributed-memory systems. It introduces batched cover-tree algorithms and two scalable strategies—point-partitioning (systolic) and spatial-partitioning (landmarking)—with a hybrid approach to maximize performance. Key contributions include efficient batched cover-tree construction and querying, ghost-point mechanisms (collective and ring) to capture cross-cell edges, and comprehensive experiments demonstrating large speedups and scalability across diverse datasets and metrics. The work enables exact, scalable near-neighbor graphs for downstream data analysis tasks on datasets with billions of edges, including non-Euclidean distances, which broadens applicability beyond Euclidean-focused methods.

Abstract

Computing fixed-radius near-neighbor graphs is an important first step for many data analysis algorithms. Near-neighbor graphs connect points that are close under some metric, endowing point clouds with a combinatorial structure. As computing power and data acquisition methods advance, diverse sources of large scientific datasets would greatly benefit from scalable solutions to this common subroutine for downstream analysis. Prior work on parallel nearest neighbors has made great progress in problems like k-nearest and approximate nearest neighbor search problems, with particular attention on Euclidean spaces. Yet many applications need exact solutions and non-Euclidean metrics. This paper presents a scalable sparsity-aware distributed memory algorithm using cover trees to compute near-neighbor graphs in general metric spaces. We provide a shared-memory algorithm for cover tree construction and demonstrate its competitiveness with state-of-the-art fixed-radius search data structures. We then introduce two distributed-memory algorithms for the near-neighbor graph problem, a simple point-partitioning strategy and a spatial-partitioning strategy, which leverage the cover tree algorithm on each node. Our algorithms exhibit parallel scaling across a variety of real and synthetic datasets for both traditional and non-traditional metrics. On real world high dimensional datasets with one million points, we achieve speedups up to 678.34x over the state-of-the-art using 1024 cores for graphs with 70 neighbors per vertex (on average), and up to 1590.99x using 4096 cores for graphs with 500 neighbors per vertex (on average).
Paper Structure (17 sections, 1 theorem, 8 equations, 5 figures, 3 tables, 6 algorithms)

This paper contains 17 sections, 1 theorem, 8 equations, 5 figures, 3 tables, 6 algorithms.

Table of Contents

  1. Introduction
  2. Related Work
  3. Definitions
  4. Algorithm
  5. Cover Tree Batch Construction
  6. Cover Tree Batch Queries
  7. Systolic Near Neighbor Graph Construction
  8. Landmarking Near Neighbor Graph Construction
  9. Ghost Points
  10. Collective Ghost Queries
  11. Ring Ghost Queries
  12. Experiments
  13. Environment
  14. Datasets
  15. Strong Scaling
  16. ...and 2 more sections

Key Result

Lemma 1

If $p \in V_j$ has an $\epsilon$-neighbor in $V_i$, where $i \neq j$, then $d(p,c_i) \leq d(p,c_j) + 2\epsilon.$

Figures (5)

  • Figure 1: Illustration of $\epsilon$-ghost points in 2D Euclidean space.
  • Figure 2: Strong Scaling
  • Figure 3: covtype landmark algorithm breakdowns: communication/synchronization times overlayed in darker colors with dotted pattern for each phase, Voronoi partitioning, tree coalescing and querying, and ghost computation and querying. Process counts where communication time begins to dominate the landmark-coll ghost query phase are shown in the top row, and the corresponding results using landmark-ring are shown in second row.
  • Figure 4: twitter landmark algorithm breakdowns: same presentation as Figure \ref{['fig:covtype_imbalance']}.
  • Figure 5: sift landmark algorithm breakdowns: same presentation as Figures \ref{['fig:covtype_imbalance']} and \ref{['fig:twitter_imbalance']}.

Theorems & Definitions (2)

  • Lemma 1
  • proof