Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

A Polynomial Space Lower Bound for Diameter Estimation in Dynamic Streams

Sanjeev Khanna, Ashwin Padaki, Krish Singal, Erik Waingarten

TL;DR

This work establishes a polynomial-space lower bound for constant-factor diameter estimation in dynamic streaming, revealing a strong separation from the insertion-only setting. The authors connect dynamic streaming to scale-invariant linear sketches and relate diameter sketches to graph minrank via index-coding-type constructions, deriving a $\tilde{\Omega}\big(n^{1/(2\lceil c\rceil-1)}\big)$ lower bound for a $c$-approximation on $\{0,1\}^n$ inputs, while also giving a matching $n^{O(1/c)}$-space upper bound via diameter embedding into $\ell_\infty$ and $\ell_0$-sampling. The paper further leverages embeddings to low-dimensional $\ell_\infty$ spaces to achieve practical upper bounds and to obtain embedding-distortion tradeoffs, enriching understanding of space–approximation barriers in dynamic geometric streams. Overall, the results delineate a fundamental limit for dynamic-diameter estimation and provide near-optimal strategies for general metrics, with implications for index coding–type hardness and metric embeddings.

Abstract

We study the space complexity of estimating the diameter of a subset of points in an arbitrary metric space in the dynamic (turnstile) streaming model. The input is given as a stream of updates to a frequency vector $x \in \mathbb{Z}_{\geq 0}^n$, where the support of $x$ defines a multiset of points in a fixed metric space $M = ([n], \mathsf{d})$. The goal is to estimate the diameter of this multiset, defined as $\max\{\mathsf{d}(i,j) : x_i, x_j > 0\}$, to a specified approximation factor while using as little space as possible. In insertion-only streams, a simple $O(\log n)$-space algorithm achieves a 2-approximation. In sharp contrast to this, we show that in the dynamic streaming model, any algorithm achieving a constant-factor approximation to diameter requires polynomial space. Specifically, we prove that a $c$-approximation to the diameter requires $n^{Ω(1/c)}$ space. Our lower bound relies on two conceptual contributions: (1) a new connection between dynamic streaming algorithms and linear sketches for {\em scale-invariant} functions, a class that includes diameter estimation, and (2) a connection between linear sketches for diameter and the {\em minrank} of graphs, a notion previously studied in index coding. We complement our lower bound with a nearly matching upper bound, which gives a $c$-approximation to the diameter in general metrics using $n^{O(1/c)}$ space.

A Polynomial Space Lower Bound for Diameter Estimation in Dynamic Streams

TL;DR

This work establishes a polynomial-space lower bound for constant-factor diameter estimation in dynamic streaming, revealing a strong separation from the insertion-only setting. The authors connect dynamic streaming to scale-invariant linear sketches and relate diameter sketches to graph minrank via index-coding-type constructions, deriving a lower bound for a -approximation on inputs, while also giving a matching -space upper bound via diameter embedding into and -sampling. The paper further leverages embeddings to low-dimensional spaces to achieve practical upper bounds and to obtain embedding-distortion tradeoffs, enriching understanding of space–approximation barriers in dynamic geometric streams. Overall, the results delineate a fundamental limit for dynamic-diameter estimation and provide near-optimal strategies for general metrics, with implications for index coding–type hardness and metric embeddings.

Abstract

We study the space complexity of estimating the diameter of a subset of points in an arbitrary metric space in the dynamic (turnstile) streaming model. The input is given as a stream of updates to a frequency vector , where the support of defines a multiset of points in a fixed metric space . The goal is to estimate the diameter of this multiset, defined as , to a specified approximation factor while using as little space as possible. In insertion-only streams, a simple -space algorithm achieves a 2-approximation. In sharp contrast to this, we show that in the dynamic streaming model, any algorithm achieving a constant-factor approximation to diameter requires polynomial space. Specifically, we prove that a -approximation to the diameter requires space. Our lower bound relies on two conceptual contributions: (1) a new connection between dynamic streaming algorithms and linear sketches for {\em scale-invariant} functions, a class that includes diameter estimation, and (2) a connection between linear sketches for diameter and the {\em minrank} of graphs, a notion previously studied in index coding. We complement our lower bound with a nearly matching upper bound, which gives a -approximation to the diameter in general metrics using space.

Paper Structure

This paper contains 38 sections, 34 theorems, 66 equations, 1 figure, 1 algorithm.

Table of Contents

  1. Introduction
  2. Our Results
  3. Related Work
  4. Technical Overview
  5. Communication Game and the Minrank.
  6. General Streaming Lower Bounds.
  7. Revisiting the Reduction in LNW14.
  8. Overview of Theorem \ref{['thm:lin-sketch-red']}: Streaming to Sketching for Scale-Invariant Problems.
  9. Overview of Theorem \ref{['thm: Diam linear sketch dim lb']}: Sketching Lower Bounds for Diameter.
  10. Organization of the Paper
  11. Preliminaries
  12. Geometric Estimation Problems
  13. Streaming Subroutine: $\ell_0$-sampling JST11
  14. Randomized Streaming Algorithms and Path-Independent Algorithms
  15. Randomized Streaming Algorithm.
  16. ...and 23 more sections

Key Result

Theorem 1

For any $c \geq 1$, there exists a metric $\mathcal{M} = ([n], \mathsf{d})$ such that any dynamic streaming algorithm that computes a $c$-approximation to $\mathrm{diam}_{\mathcal{M}}(x)$ for all $x \in \{0,1\}^n$ must use $\tilde{\Omega}(n^{1/(2\lceil {c} \rceil - 1)})$ bits of space.

Figures (1)

  • Figure 1: Diagram of the proof of Lemma \ref{['lem:relate']}. Dashed arrows denote zero-frequency transitions, blue arrows highlight the zero-frequency path from $o_3$ to $o_3^{\circ}$, and the green bidirectional arrow indicates that $o_1^{\circ}$ and $o_5$ are identical.

Theorems & Definitions (91)

  • Theorem 1
  • Theorem 2
  • Theorem 3
  • Remark 1.1
  • Theorem 3
  • Theorem 3
  • Definition 2.1: Diameter
  • Definition 2.2: Approximate Furthest Neighbor
  • Definition 2.3: $\ell_0$-sampler JST11
  • Lemma 2.4: Theorem 2 of JST11
  • ...and 81 more