Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Minimax Optimal Kernel Two-Sample Tests with Random Features

Soumya Mukherjee, Bharath K. Sriperumbudur

TL;DR

This work develops a minimax-optimal two-sample test in RKHSs by integrating spectral-regularization of the covariance operator with mean embeddings, then drastically improves scalability via a Random Fourier Features (RFF) approximation. A permutation-based, data-adaptive procedure selects the regularization parameter and, when combined with a grid over kernels, achieves minimax separation rates across polynomial and exponential eigen-decays. Theoretical results quantify the trade-off between the number of random features $l$, regularization $oldlambda$, and test power, while practical experiments (Gaussian mean/scale shifts and MNIST) demonstrate substantial computational gains with only moderate power loss. The method thus offers a scalable, flexible toolkit for nonparametric two-sample testing on general domains with strong statistical guarantees. Furthermore, the framework naturally extends to kernel- or parameter-adaptive settings and invites exploration of Nyström-type approximations for further efficiency.

Abstract

Reproducing Kernel Hilbert Space (RKHS) embedding of probability distributions has proved to be an effective approach, via MMD (maximum mean discrepancy), for nonparametric hypothesis testing problems involving distributions defined over general (non-Euclidean) domains. While a substantial amount of work has been done on this topic, only recently have minimax optimal two-sample tests been constructed that incorporate, unlike MMD, both the mean element and a regularized version of the covariance operator. However, as with most kernel algorithms, the optimal test scales cubically in the sample size, limiting its applicability. In this paper, we propose a spectral-regularized two-sample test based on random Fourier feature (RFF) approximation and investigate the trade-offs between statistical optimality and computational efficiency. We show the proposed test to be minimax optimal if the approximation order of RFF (which depends on the smoothness of the likelihood ratio and the decay rate of the eigenvalues of the integral operator) is sufficiently large. We develop a practically implementable permutation-based version of the proposed test with a data-adaptive strategy for selecting the regularization parameter. Finally, through numerical experiments on simulated and benchmark datasets, we demonstrate that the proposed RFF-based test is computationally efficient and performs almost similarly (with a small drop in power) to the exact test.

Minimax Optimal Kernel Two-Sample Tests with Random Features

TL;DR

This work develops a minimax-optimal two-sample test in RKHSs by integrating spectral-regularization of the covariance operator with mean embeddings, then drastically improves scalability via a Random Fourier Features (RFF) approximation. A permutation-based, data-adaptive procedure selects the regularization parameter and, when combined with a grid over kernels, achieves minimax separation rates across polynomial and exponential eigen-decays. Theoretical results quantify the trade-off between the number of random features , regularization , and test power, while practical experiments (Gaussian mean/scale shifts and MNIST) demonstrate substantial computational gains with only moderate power loss. The method thus offers a scalable, flexible toolkit for nonparametric two-sample testing on general domains with strong statistical guarantees. Furthermore, the framework naturally extends to kernel- or parameter-adaptive settings and invites exploration of Nyström-type approximations for further efficiency.

Abstract

Reproducing Kernel Hilbert Space (RKHS) embedding of probability distributions has proved to be an effective approach, via MMD (maximum mean discrepancy), for nonparametric hypothesis testing problems involving distributions defined over general (non-Euclidean) domains. While a substantial amount of work has been done on this topic, only recently have minimax optimal two-sample tests been constructed that incorporate, unlike MMD, both the mean element and a regularized version of the covariance operator. However, as with most kernel algorithms, the optimal test scales cubically in the sample size, limiting its applicability. In this paper, we propose a spectral-regularized two-sample test based on random Fourier feature (RFF) approximation and investigate the trade-offs between statistical optimality and computational efficiency. We show the proposed test to be minimax optimal if the approximation order of RFF (which depends on the smoothness of the likelihood ratio and the decay rate of the eigenvalues of the integral operator) is sufficiently large. We develop a practically implementable permutation-based version of the proposed test with a data-adaptive strategy for selecting the regularization parameter. Finally, through numerical experiments on simulated and benchmark datasets, we demonstrate that the proposed RFF-based test is computationally efficient and performs almost similarly (with a small drop in power) to the exact test.

Paper Structure

This paper contains 47 sections, 34 theorems, 273 equations, 10 figures, 5 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Contributions
  3. Related work
  4. Definitions, notations, and preliminaries
  5. Mean element, covariance operator, and integral operator
  6. Spectral regularization
  7. Problem setup
  8. Minimax testing
  9. Maximum mean discrepancy (MMD) test
  10. Spectral regularized MMD test
  11. Approximate spectral regularized MMD test
  12. Construction of the test statistic and the test
  13. Assumptions
  14. Oracle test
  15. Permutation test
  16. ...and 32 more sections

Key Result

Theorem 1

Suppose $\boldsymbol{(A_{0})}$--$\boldsymbol{(A_{3})}$ hold. Let $n,m \geq 2$ and $\hat{\eta}_{\lambda,l}$ be the random feature approximation of the test statistic as defined in Approximate Kernel Test statistic. Given any $\alpha>0$, suppose and Then the level-$\alpha$ critical region for testing $H_0 :P=Q$ vs. $H_1:P\neq Q$ is given by $\left\{\hat{\eta}_{\lambda,l} \geq \gamma\right\}$, i.e.

Figures (10)

  • Figure 1: Empirical power for Gaussian mean shift experiments.
  • Figure 2: Comparison of computation time (in log seconds) for Gaussian mean shift experiments.
  • Figure 3: Empirical power for Gaussian scale shift experiments.
  • Figure 4: Comparison of computation time (in log seconds) for Gaussian scale shift experiments.
  • Figure 5: Empirical power for Cauchy median shift experiments.
  • ...and 5 more figures

Theorems & Definitions (56)

  • Remark 1
  • Theorem 1: RFF-based Oracle Test
  • Theorem 2: Separation boundary of RFF-based Oracle Test
  • Corollary 3: RFF Oracle Test under polynomial decay
  • Corollary 4: RFF Oracle Test under exponential decay
  • Theorem 5: RFF-based Permutation Test
  • Theorem 6: Separation boundary of RFF-based Permutation Test
  • Corollary 7: RFF Permutation Test under polynomial decay
  • Corollary 8: RFF Permutation Test under exponential decay
  • Theorem 9: RFF-based Adaptive Test
  • ...and 46 more