Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Kernel Integrated $R^2$: A Measure of Dependence

Pouya Roudaki, Shakeel Gavioli-Akilagun, Florian Kalinke, Mona Azadkia, Zoltán Szabó

Abstract

We introduce kernel integrated $R^2$, a new measure of statistical dependence that combines the local normalization principle of the recently introduced integrated $R^2$ with the flexibility of reproducing kernel Hilbert spaces (RKHSs). The proposed measure extends integrated $R^2$ from scalar responses to responses taking values on general spaces equipped with a characteristic kernel, allowing to measure dependence of multivariate, functional, and structured data, while remaining sensitive to tail behaviour and oscillatory dependence structures. We establish that (i) this new measure takes values in $[0,1]$, (ii) equals zero if and only if independence holds, and (iii) equals one if and only if the response is almost surely a measurable function of the covariates. Two estimators are proposed: a graph-based method using $K$-nearest neighbours and an RKHS-based method built on conditional mean embeddings. We prove consistency and derive convergence rates for the graph-based estimator, showing its adaptation to intrinsic dimensionality. Numerical experiments on simulated data and a real data experiment in the context of dependency testing for media annotations demonstrate competitive power against state-of-the-art dependence measures, particularly in settings involving non-linear and structured relationships.

Kernel Integrated $R^2$: A Measure of Dependence

Abstract

We introduce kernel integrated , a new measure of statistical dependence that combines the local normalization principle of the recently introduced integrated with the flexibility of reproducing kernel Hilbert spaces (RKHSs). The proposed measure extends integrated from scalar responses to responses taking values on general spaces equipped with a characteristic kernel, allowing to measure dependence of multivariate, functional, and structured data, while remaining sensitive to tail behaviour and oscillatory dependence structures. We establish that (i) this new measure takes values in , (ii) equals zero if and only if independence holds, and (iii) equals one if and only if the response is almost surely a measurable function of the covariates. Two estimators are proposed: a graph-based method using -nearest neighbours and an RKHS-based method built on conditional mean embeddings. We prove consistency and derive convergence rates for the graph-based estimator, showing its adaptation to intrinsic dimensionality. Numerical experiments on simulated data and a real data experiment in the context of dependency testing for media annotations demonstrate competitive power against state-of-the-art dependence measures, particularly in settings involving non-linear and structured relationships.
Paper Structure (26 sections, 6 theorems, 93 equations, 2 figures, 1 table)

This paper contains 26 sections, 6 theorems, 93 equations, 2 figures, 1 table.

Table of Contents

  1. Introduction
  2. Notations
  3. General conventions.
  4. Probability measures and conditioning.
  5. Kernels and RKHS.
  6. Related Dependence Measures
  7. Kernel Integrated R2
  8. Estimation
  9. Nearest-Neighbour Estimator
  10. RKHS Estimator
  11. Computational Complexity
  12. Consistency and Rate of Convergence
  13. Numerical Illustrations
  14. Simulation Studies
  15. Power comparison.
  16. ...and 11 more sections

Key Result

Theorem 1

Under Assumption assumption:general, $D(Y, X)$ is well-defined and

Figures (2)

  • Figure 1: Comparison of power of independence tests for the heteroscedastic (left) and SO(3) (right) alternatives as a function of homoscedasticity (left) and the level of noise (right).
  • Figure 2: (Left) Comparison of power of independence tests for Million Songs Data as a function of the sample size ($n$). (Right) Comparison of time complexity of the compared dependence measures as a function of the sample size ($n$).

Theorems & Definitions (21)

  • Remark 1
  • Definition 1: Kernel integrated $R^2$
  • Remark 2
  • Theorem 1
  • Definition 2: Nearest-neighbour estimator
  • Remark 3
  • Definition 3: RKHS-based estimator
  • Remark 4
  • Remark 5
  • Theorem 2
  • ...and 11 more