Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Detecting practically significant dependencies in metric spaces via distance correlations

Holger Dette, Marius Kroll

TL;DR

This work reframes independence testing for metric-space-valued data by targeting practically significant dependencies, formalized as $H_0^{rel}: dcor(X,Y) \le \Delta$ vs $H_1^{rel}: dcor(X,Y) > \Delta$. It develops a pivotal, self-normalized, sequential test based on a functional limit theorem for the sequential distance covariance process, valid under strictly stationary $\beta$-mixing data in spaces of strong negative type, and yields a data-adaptive threshold $\hat{\Delta}_\alpha$ with controlled type I error and consistent power. The methodology provides confidence intervals for $dcor$ without resampling and extends to non-Euclidean and functional data, including time-series settings such as ARMA and GARCH models; finite-sample results demonstrate robust performance even at moderate to large dimensions. The paper also analyzes edge cases (perfect independence) and outlines avenues for extensions, such as variable importance screening and bioequivalence-type testing, highlighting substantial computational and practical benefits over resampling-based approaches.

Abstract

We take a different look at the problem of testing the independence of two metric-space-valued random variables using the distance correlation. Instead of testing if the distance correlation vanishes exactly, we are interested in the hypothesis that it does not exceed a certain threshold. Our testing problem is motivated by the observation that in many cases it is more reasonable to test for a practically significant dependency since it is rare that a hypothesis of perfect independence is exactly satisfied. This point of view also reflects statistical practice, where one often classifies the strength of the association in categories such as `small', `medium' and `large' and the precise definitions depend on the specific application. To address these problems we develop a pivotal test for the hypothesis that the distance correlation between two random variables does not exceed a pre-specified threshold $Δ$. We also determine a minimum value $\hat Δ_α$ from the data such that the hypothesis is rejected for all $Δ\leq \hat Δ_α$ at controlled type I error $α$. This quantity can be interpreted as a measure of evidence against the hypothesis that the distance correlation is less or equal than $Δ$. The new test is applicable to processes taking values in separable metric spaces of strong negative type, covering Euclidean as well as functional data. We do not assume independent observations, and instead prove our results for absolutely regular sample generating processes, which includes many time series such as ARMA and GARCH models. Our approach is based on a new functional limit theorem for the sequential distance correlation process, and can also be used to construct confidence intervals for the distance correlation without the need for resampling.

Detecting practically significant dependencies in metric spaces via distance correlations

TL;DR

This work reframes independence testing for metric-space-valued data by targeting practically significant dependencies, formalized as vs . It develops a pivotal, self-normalized, sequential test based on a functional limit theorem for the sequential distance covariance process, valid under strictly stationary -mixing data in spaces of strong negative type, and yields a data-adaptive threshold with controlled type I error and consistent power. The methodology provides confidence intervals for without resampling and extends to non-Euclidean and functional data, including time-series settings such as ARMA and GARCH models; finite-sample results demonstrate robust performance even at moderate to large dimensions. The paper also analyzes edge cases (perfect independence) and outlines avenues for extensions, such as variable importance screening and bioequivalence-type testing, highlighting substantial computational and practical benefits over resampling-based approaches.

Abstract

We take a different look at the problem of testing the independence of two metric-space-valued random variables using the distance correlation. Instead of testing if the distance correlation vanishes exactly, we are interested in the hypothesis that it does not exceed a certain threshold. Our testing problem is motivated by the observation that in many cases it is more reasonable to test for a practically significant dependency since it is rare that a hypothesis of perfect independence is exactly satisfied. This point of view also reflects statistical practice, where one often classifies the strength of the association in categories such as `small', `medium' and `large' and the precise definitions depend on the specific application. To address these problems we develop a pivotal test for the hypothesis that the distance correlation between two random variables does not exceed a pre-specified threshold . We also determine a minimum value from the data such that the hypothesis is rejected for all at controlled type I error . This quantity can be interpreted as a measure of evidence against the hypothesis that the distance correlation is less or equal than . The new test is applicable to processes taking values in separable metric spaces of strong negative type, covering Euclidean as well as functional data. We do not assume independent observations, and instead prove our results for absolutely regular sample generating processes, which includes many time series such as ARMA and GARCH models. Our approach is based on a new functional limit theorem for the sequential distance correlation process, and can also be used to construct confidence intervals for the distance correlation without the need for resampling.

Paper Structure

This paper contains 13 sections, 13 theorems, 151 equations, 7 figures, 1 table.

Table of Contents

  1. Introduction
  2. Preliminaries
  3. Distance covariance and correlation in metric spaces
  4. Absolutely regular processes
  5. Pivotal inference for distance covariance and correlation
  6. Behaviour Under Independence
  7. Finite Sample Properties
  8. Conclusions
  9. Proofs
  10. Preliminaries and Proofs of Theorem \ref{['thm:dcov_prozesskonvergenz']} and Corollary \ref{['cor:dcor_processkonvergenz']}
  11. Proofs of Theorems \ref{['thm:schwache_konvergenz_dcov']} and \ref{['thm:schwache_konvergenz_dcor']}
  12. Proof of Proposition \ref{['cor:test_dcov']}
  13. Proof of Theorem \ref{['thm:independence_weak_convergence']}

Key Result

Theorem 1

For each $\lambda \in [0,1]$ define the random vector and consider the processes $X_n := (X_n(\lambda))_{0 \leq \lambda \leq 1}$ and $X := (X(\lambda))_{0 \leq \lambda \leq 1}$. Suppose that $X$ and $Y$ are not almost surely constant and have finite $(4+\varepsilon)$-moments for some $\varepsilon > 0$. Assume further that the sample generating process in $(\ell^\infty[0,1])^3$, where $W = (B_1,

Figures (7)

  • Figure 1: An illustration of the visual inspection method described in Remark \ref{['rem:visual_inspection']} to determine whether $\tau > 0$. The plots show the sample paths of the process on the left-hand side of Eq. \ref{['eq:visual_inspection']} under model \ref{['eq:var_model']} for two different values of $\rho$. The two panels are examples of cases where $\tau > 0$ (left panel) and where $\tau \approx 0$ (right panel), and even at small sample sizes these two cases can clearly be distinguished. The values of $\tau$ given in the captions of the plots were obtained by Monte Carlo simulations.
  • Figure 2: Simulation results for the model \ref{['eq:var_model']} with varying parameter $\rho$. The $x$-axes give the corresponding values for $\mathrm{dcor}(X,Y)$. Empirical rejection rates of the test \ref{['det5']} for the hypotheses \ref{['det2']} for different thresholds $\Delta$ and sample sizes $n$. The dotted lines represent $\Delta$ (vertical) and the nominal level $\alpha = 5\%$ (horizontal).
  • Figure 3: Empirical covering rates of the confidence interval \ref{['eq:confidence_interval']} with $\alpha = 0.95$ and data generated according to Eq. \ref{['eq:var_model']}. The dashed line represents the nominal level of $95 \%$.
  • Figure 4: Empirical rejection rates of the test \ref{['det33']} for the hypotheses \ref{['det3']} for different thresholds $\Delta$ and sample sizes $n$. The dotted lines represent $\Delta$ (vertical) and the nominal level $\alpha = 5\%$ (horizontal). The data are simulated according to the VAR model \ref{['eq:var_model']}.
  • Figure 5: Simulation results for the model \ref{['eq:fourier_model']} with covariance matrix \ref{['eq:sparse']} and varying parameter $\rho$. The $x$-axes give the corresponding values for $\mathrm{dcor}(X,Y)$. Empirical rejection rates of the test \ref{['det5']} for the hypotheses \ref{['det2']} for different thresholds $\Delta$ and sample sizes $n$. The dotted lines represent $\Delta$ (vertical) and the nominal level $\alpha = 5\%$ (horizontal).
  • ...and 2 more figures

Theorems & Definitions (29)

  • Theorem 1
  • Corollary 1
  • Theorem 2
  • Theorem 3
  • Proposition 1
  • Remark 1
  • Remark 2
  • Remark 3
  • Remark 4
  • Theorem 4
  • ...and 19 more