Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Testing vs Estimation for Index-Invariant Properties in the Huge Object Model

Sourav Chakraborty, Eldar Fischer, Arijit Ghosh, Amit Levi, Gopinath Mishra, Sayantan Sen

TL;DR

This work extends property testing in the Huge Object model to index-invariant properties by importing Szemerédi-style regularity ideas into a detailing framework. It shows that for index-invariant properties that admit a constant-Query $\varepsilon$-test, one can construct a distance-estimation algorithm with constant-query complexity, effectively bridging testing and tolerant estimation in this setting. The approach introduces detailings, weight/type distributions, and robust/weakt robustness notions, and builds a simulation-based pipeline (Simulate, Accept-Probability) together with procedures to locate robust detailings and estimate their parameters. The results highlight a sharp separation enabled by invariance: constant-query testability implies constant-query estimability, with controlled (non-tower) parameter dependencies, offering a principled regularity-like framework for Huge Object property testing.

Abstract

The Huge Object model of property testing [Goldreich and Ron, TheoretiCS 23] concerns properties of distributions supported on $\{0,1\}^n$, where $n$ is so large that even reading a single sampled string is unrealistic. Instead, query access is provided to the samples, and the efficiency of the algorithm is measured by the total number of queries that were made to them. Index-invariant properties under this model were defined in [Chakraborty et al., COLT 23], as a compromise between enduring the full intricacies of string testing when considering unconstrained properties, and giving up completely on the string structure when considering label-invariant properties. Index-invariant properties are those that are invariant through a consistent reordering of the bits of the involved strings. Here we provide an adaptation of Szemerédi's regularity method for this setting, and in particular show that if an index-invariant property admits an $ε$-test with a number of queries depending only on the proximity parameter $ε$, then it also admits a distance estimation algorithm whose number of queries depends only on the approximation parameter.

Testing vs Estimation for Index-Invariant Properties in the Huge Object Model

TL;DR

This work extends property testing in the Huge Object model to index-invariant properties by importing Szemerédi-style regularity ideas into a detailing framework. It shows that for index-invariant properties that admit a constant-Query -test, one can construct a distance-estimation algorithm with constant-query complexity, effectively bridging testing and tolerant estimation in this setting. The approach introduces detailings, weight/type distributions, and robust/weakt robustness notions, and builds a simulation-based pipeline (Simulate, Accept-Probability) together with procedures to locate robust detailings and estimate their parameters. The results highlight a sharp separation enabled by invariance: constant-query testability implies constant-query estimability, with controlled (non-tower) parameter dependencies, offering a principled regularity-like framework for Huge Object property testing.

Abstract

The Huge Object model of property testing [Goldreich and Ron, TheoretiCS 23] concerns properties of distributions supported on , where is so large that even reading a single sampled string is unrealistic. Instead, query access is provided to the samples, and the efficiency of the algorithm is measured by the total number of queries that were made to them. Index-invariant properties under this model were defined in [Chakraborty et al., COLT 23], as a compromise between enduring the full intricacies of string testing when considering unconstrained properties, and giving up completely on the string structure when considering label-invariant properties. Index-invariant properties are those that are invariant through a consistent reordering of the bits of the involved strings. Here we provide an adaptation of Szemerédi's regularity method for this setting, and in particular show that if an index-invariant property admits an -test with a number of queries depending only on the proximity parameter , then it also admits a distance estimation algorithm whose number of queries depends only on the approximation parameter.

Paper Structure

This paper contains 34 sections, 64 theorems, 123 equations, 8 figures.

Table of Contents

  1. Introduction
  2. Technical overview
  3. Detailings
  4. Goodness and predictability
  5. Robustness and inheritance
  6. Distance estimation algorithm
  7. Relation to prior works
  8. Organization of the paper
  9. Preliminaries
  10. Basic handling and manipulation of distributions
  11. Distribution distances
  12. The testing model
  13. Canonical testers
  14. Quantized distributions and distance bounds
  15. Some useful probabilistic inequalities
  16. ...and 19 more sections

Key Result

Theorem 1

If $\mathcal{P}$ is an index-invariant property of distributions over $\{0,1\}^n$ which is $\epsilon'$-testable with a constant number of queries for any fixed $\epsilon'>0$, then there is an estimation algorithm with a constant number of queries for any fixed $\epsilon>0$, that for a distribution

Figures (8)

  • Figure 1: Description of the $\texttt{Simulate}$ procedure.
  • Figure 2: Description of the Accept-Probability procedure.
  • Figure 3: Description of the $\textsf{Estimate-Index}$ procedure.
  • Figure 4: A description of the Test-Weakly-Robust-Detailing procedure.
  • Figure 5: A description of the Find-Weakly-Robust-Detailing procedure.
  • ...and 3 more figures

Theorems & Definitions (111)

  • Theorem 1: Informal statement of Theorem \ref{['thm:main-estimation']}
  • Definition 2.1: Restriction of a distribution
  • Definition 2.2: Projection of a distribution
  • Definition 2.3: Conditioning shorthand
  • Definition 2.4: Join of distributions
  • Definition 2.5: Adjustment
  • Definition 2.6: Variation distance
  • Definition 2.7: Transfer distribution
  • Definition 2.8: EMD with respect to a distance function
  • Lemma 2.11
  • ...and 101 more