Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Unified Unbiased Variance Estimation for MMD: Robust Finite-Sample Performance with Imbalanced Data and Exact Acceleration under Null and Alternative Hypotheses

Shijie Zhong, Jiangfeng Fu, Yikun Yang

TL;DR

This work provides a unified, finite-sample variance characterization for the Maximum Mean Discrepancy (MMD) that holds under both null and alternative hypotheses and across balanced or imbalanced sample configurations, by leveraging the Hoeffding decomposition of U-statistics. It derives a practical, unbiased estimator of the MMD variance that decomposes into first-order and second-order components, with explicit, numerically stable estimators that handle independence between samples. A key contribution is an exact, near-linear-time acceleration for univariate MMD with the Laplace kernel (euMMD), achieved via prefix-suffix recurrences and sorted prefix sums, reducing complexity from $O(n^2)$ to $O(n \, \log n)$. The proposed framework improves robustness to sample imbalance and provides a transparent decomposition of variance, including a tractable second-order term, with strong empirical validation showing accurate variance estimation and substantial computational gains for large-scale data.

Abstract

The maximum mean discrepancy (MMD) is a kernel-based nonparametric statistic for two-sample testing, whose inferential accuracy depends critically on variance characterization. Existing work provides various finite-sample estimators of the MMD variance, often differing under the null and alternative hypotheses and across balanced or imbalanced sampling schemes. In this paper, we study the variance of the MMD statistic through its U-statistic representation and Hoeffding decomposition, and establish a unified finite-sample characterization covering different hypotheses and sample configurations. Building on this analysis, we propose an exact acceleration method for the univariate case under the Laplacian kernel, which reduces the overall computational complexity from $\mathcal O(n^2)$ to $\mathcal O(n \log n)$.

Unified Unbiased Variance Estimation for MMD: Robust Finite-Sample Performance with Imbalanced Data and Exact Acceleration under Null and Alternative Hypotheses

TL;DR

This work provides a unified, finite-sample variance characterization for the Maximum Mean Discrepancy (MMD) that holds under both null and alternative hypotheses and across balanced or imbalanced sample configurations, by leveraging the Hoeffding decomposition of U-statistics. It derives a practical, unbiased estimator of the MMD variance that decomposes into first-order and second-order components, with explicit, numerically stable estimators that handle independence between samples. A key contribution is an exact, near-linear-time acceleration for univariate MMD with the Laplace kernel (euMMD), achieved via prefix-suffix recurrences and sorted prefix sums, reducing complexity from to . The proposed framework improves robustness to sample imbalance and provides a transparent decomposition of variance, including a tractable second-order term, with strong empirical validation showing accurate variance estimation and substantial computational gains for large-scale data.

Abstract

The maximum mean discrepancy (MMD) is a kernel-based nonparametric statistic for two-sample testing, whose inferential accuracy depends critically on variance characterization. Existing work provides various finite-sample estimators of the MMD variance, often differing under the null and alternative hypotheses and across balanced or imbalanced sampling schemes. In this paper, we study the variance of the MMD statistic through its U-statistic representation and Hoeffding decomposition, and establish a unified finite-sample characterization covering different hypotheses and sample configurations. Building on this analysis, we propose an exact acceleration method for the univariate case under the Laplacian kernel, which reduces the overall computational complexity from to .
Paper Structure (28 sections, 7 theorems, 64 equations, 3 figures, 7 tables, 5 algorithms)

This paper contains 28 sections, 7 theorems, 64 equations, 3 figures, 7 tables, 5 algorithms.

Table of Contents

  1. Introduction
  2. Backgroud
  3. Maximum Mean Discrepancy
  4. U-statistics and Hoeffding Decomposition
  5. Lossless acceleration of the unbiased MMD estimator with the Laplace kernel
  6. Unbiased estimator of Maximum Mean Discrepancy variance
  7. Variance derivation of the unbiased MMD estimator
  8. Simplification and unbiased estimation of the variance
  9. Simplification and unbiased estimation of first-order projection terms
  10. Simplification and unbiased estimation of second-order residual terms
  11. The unbiased estimator of MMD variance
  12. Accelerated computation of Maximum Mean Discrepancy
  13. Several Propositions and Lemma for acceleration
  14. Accelerated computation of variance components
  15. Accelerated computation of first-order components.
  16. ...and 13 more sections

Key Result

Proposition 1

Let $\{x_1,\dots,x_n\}\subset\mathbb{R}$ be sorted such that $x_1 \le x_2 \le \dots \le x_n$, and consider the Laplacian kernel. Define $R_1=L_n=0$. For $i=2,\dots,n$ and $i=n-1,\dots,1$, respectively, let Then, for each $i=1,\dots,n$, the off-diagonal row sum of the Laplace kernel matrix satisfies

Figures (3)

  • Figure 1: Empirical sample distributions in different cases.
  • Figure 2: Growth trend of the variance under increasing distribution shift.
  • Figure 3: Double-logarithmic comparison of runtime and memory consumption for both algorithms under unequal sample sizes

Theorems & Definitions (10)

  • Remark 1
  • Proposition 1
  • Proposition 2
  • Remark 2
  • Lemma 3
  • proof
  • Lemma 1
  • Lemma 2
  • Lemma 3
  • Proposition 4