Kernel conditional tests from learning-theoretic bounds

TL;DR

This work develops a general framework for hypothesis testing on conditional distributions that yields finite-sample, covariate-specific guarantees by converting learning-confidence bounds into statistical tests. It instantiates the framework with kernel ridge regression (KRR) to obtain time-uniform, vector-valued bounds that extend to infinite-dimensional outputs via uniform-block-diagonal kernels (UBD) and kernel mean embeddings. A key contribution is a principled link between estimation accuracy and conditional testing, enabling conditional two-sample tests for functionals of conditional distributions through representation maps and KMEs. To enhance practical applicability, the authors introduce bootstrapping schemes for testing thresholds and demonstrate the approach on process-monitoring and dynamical-system comparison tasks, showing localized power and robustness to online sampling. Overall, the paper provides a comprehensive theory-to-practice pipeline for conditional testing with strong guarantees and broad applicability in non-iid, online settings.

Abstract

We propose a framework for hypothesis testing on conditional probability distributions, which we then use to construct statistical tests of functionals of conditional distributions. These tests identify the inputs where the functionals differ with high probability, and include tests of conditional moments or two-sample tests. Our key idea is to transform confidence bounds of a learning method into a test of conditional expectations. We instantiate this principle for kernel ridge regression (KRR) with subgaussian noise. An intermediate data embedding then enables more general tests -- including conditional two-sample tests -- via kernel mean embeddings of distributions. To have guarantees in this setting, we generalize existing pointwise-in-time or time-uniform confidence bounds for KRR to previously-inaccessible yet essential cases such as infinite-dimensional outputs with non-trace-class kernels. These bounds also circumvent the need for independent data, allowing for instance online sampling. To make our tests readily applicable in practice, we introduce bootstrapping schemes leveraging the parametric form of testing thresholds identified in theory to avoid tuning inaccessible parameters. We illustrate the tests on examples, including one in process monitoring and comparison of dynamical systems. Overall, our results establish a comprehensive foundation for conditional testing on functionals, from theoretical guarantees to an algorithmic implementation, and advance the state of the art on confidence bounds for vector-valued least squares estimation.

Figures (8)

• Figure 1: Analytical and bootstrapped tests on a toy example. The tests reject $H_0$ (green thick line) when the confidence intervals (shaded regions) do not overlap, and accept it (red thick line) otherwise.
• Figure 2: Empirical type $\mathrm{I}$ or $\mathrm{II}$ errors. Left: Both tests uphold and achieve the level under $H_0$. Middle: Larger difference $\xi$ between conditional expectations decrease the type $\mathrm{II}$ error. Right: Our local test can confidently detect a difference in rarely (probability $\theta$) sampled regions.
• Figure 3: Left: Ratio between test statistic and threshold for the process monitoring example for various dimensions ($d$) and perturbation magnitudes ($\xi$). Triggers correspond to exceeding $1$. Solid lines are means; shaded regions are $5\%$ and $95\%$ quantiles. Right: Ratio of $\sigma_{D,\lambda}(x)$ in \ref{['eq:definition variance']} on the reference data $D$ between $x$ in the window and the initial state at $t=0$, averaged over the window.
• Figure 4: Comparing the Gaussian and inhomogeneous linear kernels on different distributions having the same conditional mean on the measurement set $\mathcal{Z}$. A trigger is a true positive for the Gaussian kernel, and a false positive for the linear one. The results are averaged over 100 runs of the experiment, with the shaded regions reporting the 2.5% and 97.5% quantiles.
• Figure 5: Empirical type $\mathrm{I}$ and $\mathrm{II}$ errors for different output kernels. For the type $\mathrm{II}$ error, conditional expectations in $\mathcal{Z}$ differ but higher moments coincide. We see that Gaussian kernels are more conservative and achieve higher type $\mathrm{II}$ error. The results are averaged over 100 runs of the experiment, with the shaded regions reporting the 2.5% and 97.5% quantiles.

Theorems & Definitions (47)

• Definition 2.1
• Definition 2.2
• Theorem 2.3
• Definition 2.4
• Definition 3.1
• Definition 3.2
• Remark 4.1
• Theorem 4.2
• Theorem 4.3
• Theorem 4.4