Robust Kernel Hypothesis Testing under Data Corruption

TL;DR

Robust Kernel Hypothesis Testing under Data Corruption addresses hypothesis testing when data can be adversarially corrupted by up to $r$ samples. It introduces the DC procedure, a general framework that preserves non-asymptotic type I error control by adjusting permutation thresholds using the statistic's global sensitivity, and applies it to kernel-based two-sample (dcMMD) and independence (dcHSIC) testing. The authors prove non-asymptotic validity, consistency, and minimax optimal separation rates in terms of MMD and HSIC, with favorable behavior in both low and high corruption regimes. They also compare with differential-privacy-based methods, showing higher power for DC while maintaining robustness, and provide public code to facilitate practical deployment in corrupt data scenarios.

Abstract

We propose a general method for constructing robust permutation tests under data corruption. The proposed tests effectively control the non-asymptotic type I error under data corruption, and we prove their consistency in power under minimal conditions. This contributes to the practical deployment of hypothesis tests for real-world applications with potential adversarial attacks. For the two-sample and independence settings, we show that our kernel robust tests are minimax optimal, in the sense that they are guaranteed to be non-asymptotically powerful against alternatives uniformly separated from the null in the kernel MMD and HSIC metrics at some optimal rate (tight with matching lower bound). We point out that existing differentially private tests can be adapted to be robust to data corruption, and we demonstrate in experiments that our proposed tests achieve much higher power than these private tests. Finally, we provide publicly available implementations and empirically illustrate the practicality of our robust tests.

Figures (2)

• Figure 1: Two-sample experiments robust up to $r$ corrupted samples. To have valid level, a robust test needs to control the rejection rate by $\alpha=0.05$ when fewer than $r$ samples are corrupted. To be powerful, the robust test needs to have a high rejection rate when more than $r$ samples are corrupted. (Top row: Gaussian mean shift) Both samples are originally i.i.d. drawn from $\mathsf{Gaussian}(0,1/10,50)$, entries of one sample are corrupted being replaced by samples from $\mathsf{Gaussian}(1000,1/10,50)$. (Bottom row: IMDb movie reviews) Both samples originally consist of movie reviews (using a bag of 3330 words representation). Corrupted entries for one sample are replaced by samples from $\mathsf{Geometric}(0.05,3330)$.
• Figure 2: Independence experiments robust up to $r$ corrupted samples. To have valid level, a robust test needs to control the rejection rate by $\alpha=0.05$ when fewer than $r$ samples are corrupted. To be powerful, the robust test needs to have a high rejection rate when more than $r$ samples are corrupted. (Top row: Gaussian mixture) Paired samples $(X,Y)$ are originally i.i.d. drawn from two $\mathsf{Gaussian}(0,1/10,50)$. Corrupted samples are replaced by $(X, X + \epsilon)$ where $\epsilon \sim \mathsf{Gaussian}(0,1/10,50)$ and where $X\sim\mathsf{Gaussian}(s1000,1/10,50)$ with $s=1$ for half of the corrupted samples and $s=-1$ for the other half. (Bottom row: IMDb movie reviews) Paired samples $(X,Y)$ originally consist of two independent reviews (represented using a bag of 3330 words). Corrupted samples are replaced by $(X + s, X + s + \epsilon)$ where $X\sim\mathsf{Geometric}(0.05,3330)$, $\epsilon \sim \mathsf{Gaussian}(0, 1/10,3330)$ and with $s=0$ for half of the corrupted samples and $s=5$ for the other half.

Theorems & Definitions (25)

• Lemma 1: Validity & consistency of DC
• Lemma 2: Consistency of dcMMD
• Theorem 1: Minimax optimal uniform separation of dcMMD
• Lemma 3: Consistency of dcHSIC
• Theorem 2: Minimax optimal uniform separation of dcHSIC
• Lemma 3: Validity & consistency of DC
• proof
• Lemma 3: Consistency of dcMMD
• proof
• Theorem 2: Minimax optimal uniform separation of dcMMD