Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

The Minimax Lower Bound of Kernel Stein Discrepancy Estimation

Jose Cribeiro-Ramallo, Agnideep Aich, Florian Kalinke, Ashit Baran Aich, Zoltán Szabó

TL;DR

This work addresses the fundamental problem of estimating Kernel Stein Discrepancies (KSDs) between a known target distribution and a sampling distribution. It establishes a minimax lower bound of $n^{-1/2}$ for KSD estimation, matching the rates of existing estimators and thereby proving their optimality. The authors present two complementary proofs: one for the Langevin-Stein KSD on $\mathbb{R}^d$ with translation-invariant, bounded, characteristic kernels (explicitly giving Gaussian-kernel constants that grow exponentially with dimension), and a second for KSD on general domains using a broad, weak-validity framework. Overall, the results quantify the intrinsic difficulty of KSD estimation and confirm that current estimators cannot be improved in rate, while highlighting dimensionality’s sharp impact on difficulty.

Abstract

Kernel Stein discrepancies (KSDs) have emerged as a powerful tool for quantifying goodness-of-fit over the last decade, featuring numerous successful applications. To the best of our knowledge, all existing KSD estimators with known rate achieve $\sqrt n$-convergence. In this work, we present two complementary results (with different proof strategies), establishing that the minimax lower bound of KSD estimation is $n^{-1/2}$ and settling the optimality of these estimators. Our first result focuses on KSD estimation on $\mathbb R^d$ with the Langevin-Stein operator; our explicit constant for the Gaussian kernel indicates that the difficulty of KSD estimation may increase exponentially with the dimensionality $d$. Our second result settles the minimax lower bound for KSD estimation on general domains.

The Minimax Lower Bound of Kernel Stein Discrepancy Estimation

TL;DR

This work addresses the fundamental problem of estimating Kernel Stein Discrepancies (KSDs) between a known target distribution and a sampling distribution. It establishes a minimax lower bound of for KSD estimation, matching the rates of existing estimators and thereby proving their optimality. The authors present two complementary proofs: one for the Langevin-Stein KSD on with translation-invariant, bounded, characteristic kernels (explicitly giving Gaussian-kernel constants that grow exponentially with dimension), and a second for KSD on general domains using a broad, weak-validity framework. Overall, the results quantify the intrinsic difficulty of KSD estimation and confirm that current estimators cannot be improved in rate, while highlighting dimensionality’s sharp impact on difficulty.

Abstract

Kernel Stein discrepancies (KSDs) have emerged as a powerful tool for quantifying goodness-of-fit over the last decade, featuring numerous successful applications. To the best of our knowledge, all existing KSD estimators with known rate achieve -convergence. In this work, we present two complementary results (with different proof strategies), establishing that the minimax lower bound of KSD estimation is and settling the optimality of these estimators. Our first result focuses on KSD estimation on with the Langevin-Stein operator; our explicit constant for the Gaussian kernel indicates that the difficulty of KSD estimation may increase exponentially with the dimensionality . Our second result settles the minimax lower bound for KSD estimation on general domains.
Paper Structure (21 sections, 17 theorems, 79 equations)

This paper contains 21 sections, 17 theorems, 79 equations.

Table of Contents

  1. INTRODUCTION
  2. NOTATIONS
  3. KERNEL STEIN DISCREPANCY
  4. Langevin-Stein KSD on Rd
  5. General KSD
  6. KSD ESTIMATORS
  7. MINIMAX ESTIMATION
  8. RESULTS
  9. Langevin-Stein KSD
  10. General KSD
  11. PROOF SKETCHES
  12. Proof Sketch for Theorem \ref{['main:thm:minimax']}
  13. Proof Sketch for Theorem \ref{['main:thm:minimax-lower']}
  14. PROOFS
  15. Proof of Lemma \ref{['lemma:lsksd-closed-form']}
  16. ...and 6 more sections

Key Result

Theorem 1

Suppose that Assumptions main:ass:LS-KSD and main:ass:kernel_k_LS hold, and that $k$ is characteristic. Let $\hat{F}_n$ be any estimator of $\mathop{\mathrm{KSD}}\nolimits(P_0,P)$ using $n \in \mathbb N_{>0}$ samples from $P \in \mathcal{S}_{P_0}$ ($P_0 \in \mathcal{T}$), where $\mathcal{S}_{P_0}$ i with $\hat{\Delta}_n$ as defined in main:eq:markov-reduction. In particular, by main:eq:markov-redu

Theorems & Definitions (23)

  • Theorem 1: minimax lower bound of Langevin-Stein KSD
  • Remark 1
  • Corollary 1
  • Theorem 2: minimax lower bound of general KSD
  • Remark 2
  • Theorem 3: Theorem 2.2; tsybakov09introduction
  • Lemma 1: KSD in terms of characteristic functions
  • Lemma 2: Lemma \ref{['main:lemma:lsksd-closed-form']} with $P=\mathcal{N}(\bm \mu, \bm \Sigma)$
  • Lemma B.1: Lebesgue integrability of key functions
  • proof
  • ...and 13 more